-- Logs begin at Thu 2022-10-27 08:57:46 UTC, end at Thu 2022-10-27 09:55:56 UTC. -- Oct 27 09:22:58.591684 dv-cinder-797970-7-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 27 09:22:58.600433 dv-cinder-797970-7-iscsi devstack@keystone.service[81423]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 27 09:22:58.602459 dv-cinder-797970-7-iscsi devstack@keystone.service[81423]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Oct 27 09:22:58 2022] *** Oct 27 09:22:58.602459 dv-cinder-797970-7-iscsi devstack@keystone.service[81423]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 27 09:22:58.602459 dv-cinder-797970-7-iscsi devstack@keystone.service[81423]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 27 09:22:58.602459 dv-cinder-797970-7-iscsi devstack@keystone.service[81423]: nodename: dv-cinder-797970-7-iscsi Oct 27 09:22:58.602459 dv-cinder-797970-7-iscsi devstack@keystone.service[81423]: machine: x86_64 Oct 27 09:22:58.602459 dv-cinder-797970-7-iscsi devstack@keystone.service[81423]: clock source: unix Oct 27 09:22:58.602459 dv-cinder-797970-7-iscsi devstack@keystone.service[81423]: pcre jit disabled Oct 27 09:22:58.602459 dv-cinder-797970-7-iscsi devstack@keystone.service[81423]: detected number of CPU cores: 4 Oct 27 09:22:58.602459 dv-cinder-797970-7-iscsi devstack@keystone.service[81423]: current working directory: / Oct 27 09:22:58.602459 dv-cinder-797970-7-iscsi devstack@keystone.service[81423]: detected binary path: /usr/bin/uwsgi-core Oct 27 09:22:58.602459 dv-cinder-797970-7-iscsi devstack@keystone.service[81423]: your processes number limit is 47680 Oct 27 09:22:58.603097 dv-cinder-797970-7-iscsi devstack@keystone.service[81423]: your memory page size is 4096 bytes Oct 27 09:22:58.603097 dv-cinder-797970-7-iscsi devstack@keystone.service[81423]: detected max file descriptor number: 2048 Oct 27 09:22:58.603097 dv-cinder-797970-7-iscsi devstack@keystone.service[81423]: lock engine: pthread robust mutexes Oct 27 09:22:58.603097 dv-cinder-797970-7-iscsi devstack@keystone.service[81423]: thunder lock: enabled Oct 27 09:22:58.603097 dv-cinder-797970-7-iscsi devstack@keystone.service[81423]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 27 09:22:58.603097 dv-cinder-797970-7-iscsi devstack@keystone.service[81423]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 27 09:22:58.641972 dv-cinder-797970-7-iscsi devstack@keystone.service[81423]: Python main interpreter initialized at 0x561472b829c0 Oct 27 09:22:58.641972 dv-cinder-797970-7-iscsi devstack@keystone.service[81423]: python threads support enabled Oct 27 09:22:58.641972 dv-cinder-797970-7-iscsi devstack@keystone.service[81423]: your server socket listen backlog is limited to 100 connections Oct 27 09:22:58.641972 dv-cinder-797970-7-iscsi devstack@keystone.service[81423]: your mercy for graceful operations on workers is 90 seconds Oct 27 09:22:58.642340 dv-cinder-797970-7-iscsi devstack@keystone.service[81423]: mapped 403077 bytes (393 KB) for 2 cores Oct 27 09:22:58.642418 dv-cinder-797970-7-iscsi devstack@keystone.service[81423]: *** Operational MODE: preforking *** Oct 27 09:22:58.642652 dv-cinder-797970-7-iscsi devstack@keystone.service[81423]: *** uWSGI is running in multiple interpreter mode *** Oct 27 09:22:58.642652 dv-cinder-797970-7-iscsi devstack@keystone.service[81423]: spawned uWSGI master process (pid: 81423) Oct 27 09:22:58.642847 dv-cinder-797970-7-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 27 09:22:58.645406 dv-cinder-797970-7-iscsi devstack@keystone.service[81423]: spawned uWSGI worker 1 (pid: 81436, cores: 1) Oct 27 09:22:58.645406 dv-cinder-797970-7-iscsi devstack@keystone.service[81423]: spawned uWSGI worker 2 (pid: 81437, cores: 1) Oct 27 09:22:58.645406 dv-cinder-797970-7-iscsi devstack@keystone.service[81423]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 27 09:23:02.883890 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 27 09:23:02.883890 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: from cryptography.utils import int_from_bytes Oct 27 09:23:02.884693 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 27 09:23:02.884693 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: from cryptography.utils import int_from_bytes Oct 27 09:23:02.885038 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 27 09:23:02.885038 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: from cryptography.utils import int_from_bytes Oct 27 09:23:02.885499 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 27 09:23:02.885499 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: from cryptography.utils import int_from_bytes Oct 27 09:23:02.934113 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 27 09:23:02.934296 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 27 09:23:02.934483 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] command line args: [] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 27 09:23:02.934746 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 27 09:23:02.934918 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ================================================================================ {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 27 09:23:02.935168 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] admin_token = **** {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.935408 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] config_dir = [] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.935493 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 27 09:23:02.935594 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.935681 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 27 09:23:02.935759 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] config_source = [] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.935836 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] command line args: [] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 27 09:23:02.935918 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] debug = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.936054 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 27 09:23:02.936137 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.936364 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ================================================================================ {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 27 09:23:02.936495 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.936653 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] admin_token = **** {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.936762 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.936867 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] config_dir = [] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.936972 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.937078 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.937078 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] config_source = [] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.937288 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.937393 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] debug = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.937492 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] list_limit = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.937588 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.937717 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] log_config_append = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.937717 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.937914 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.938012 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] log_dir = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.938120 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.938218 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] log_file = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.938322 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.938432 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.938512 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.938632 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.938714 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] list_limit = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.938837 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.938915 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] log_config_append = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.938915 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.939107 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.939188 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] log_dir = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.939306 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.939389 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] log_file = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.939516 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.939594 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.939715 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.939797 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.939875 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.939999 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.940086 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.940086 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.940225 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.940297 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.940419 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.940574 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.940678 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.940754 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.940826 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.940951 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.941023 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.941142 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.941231 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.941377 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.941609 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.941711 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.70/identity {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.941843 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.941931 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] publish_errors = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.942090 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.942168 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.942314 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.942439 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.942564 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.942679 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.942794 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.942931 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.943074 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.70/identity {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.943203 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] standard_threads = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.943325 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] publish_errors = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.943447 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.943583 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.943707 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.943824 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.943939 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.944060 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.944178 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] use_journal = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.944178 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] use_json = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.944409 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.944544 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] use_stderr = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.944664 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.944790 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] use_syslog = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.944914 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] standard_threads = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.945100 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.945221 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.945337 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.945462 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.945580 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.945701 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.945827 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.945944 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] use_journal = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.945944 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] use_json = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.946158 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.946274 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] use_stderr = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.946389 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.946516 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] use_syslog = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.946638 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.946760 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 09:23:02.946877 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.946998 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.947120 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.947247 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.947371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.947492 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.947609 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.947724 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.947839 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.947966 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.948083 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.948197 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.948313 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.948429 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.948558 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.948691 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.948804 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.949461 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.949582 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.949705 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.949830 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.950036 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.950177 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.950343 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.950466 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.950587 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.950704 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.950828 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.950945 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.951087 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.951205 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.951321 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.951443 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.951649 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.951768 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.951890 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.952007 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.952007 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.952265 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.952388 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.952505 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.952677 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.952801 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.952918 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.953082 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.953237 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.953356 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.953471 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.953588 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.953703 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.953827 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.953944 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.954059 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.954174 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.954289 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.954403 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.954527 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.954650 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.954773 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.954889 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.955004 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.955120 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.955247 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.955364 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.955486 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.955607 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.955724 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.955839 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.955962 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.956078 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.956193 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.956309 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.956424 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.956552 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.956682 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.956801 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.956917 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.957032 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.957147 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.957262 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.957262 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.957557 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.957673 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.957789 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.957905 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.958021 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.958144 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.958260 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.958376 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.958490 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.958611 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.958733 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.958859 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.958975 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.959090 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.959205 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.959327 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.959449 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.959557 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.959655 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.959752 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.959849 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.959947 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.960052 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.960150 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.960253 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.960351 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.960484 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.960649 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.960766 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.960884 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.961001 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.961126 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.961244 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.961359 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.961475 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.961600 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.961708 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.961823 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.961938 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.962045 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.962169 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.962287 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.962405 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.962521 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.962637 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.962753 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.962870 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.962994 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.963121 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.963239 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.963355 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.963483 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.963591 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.963706 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.963831 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.963948 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.964064 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.964181 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.964303 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.964421 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.964562 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.964688 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.964805 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.964921 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.965027 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.965148 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.965264 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.965380 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.965500 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.965617 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.965734 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.965850 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.965977 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.966095 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.966212 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.966334 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.966451 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.966567 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.966691 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.966815 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.966931 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.967048 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.967170 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.967288 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.967413 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.967531 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.967646 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.967763 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.967879 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.967995 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.968118 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.968236 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.968361 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.968482 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.968482 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.968743 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.968938 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.969058 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.969174 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.969297 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.969420 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.969546 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.969672 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.969796 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.969914 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.970031 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.970153 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.970270 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.970645 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.970775 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.970900 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.971017 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.971382 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.971503 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.971620 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.971745 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.971863 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.971980 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.972109 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] auth.external = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.972228 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.972344 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.972469 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.972616 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.972736 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.972852 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.972969 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.973112 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] auth.password = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.973243 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.973362 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] auth.token = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.973487 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.973606 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.973724 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.973841 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.973958 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.974075 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.974204 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.974322 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.974439 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] auth.external = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.974556 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.974673 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.974790 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.974916 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.975035 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.975158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.975277 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] credential.caching = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.975401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] auth.password = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.975515 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.975642 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] auth.token = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.975760 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.975877 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.976033 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.976150 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.976267 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.976392 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.976511 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.976674 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.976802 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.976922 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.977039 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.977164 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.977283 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.977406 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.977524 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.977641 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] credential.caching = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.977759 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.977884 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.978003 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.978120 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.978237 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.978354 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.978471 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.978601 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.978719 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.978837 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.978955 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.979078 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.979196 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] federation.caching = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.979322 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.979446 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.979564 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.979682 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.979800 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.979918 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.980044 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.980163 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.980310 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.980429 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.980429 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.980678 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.980797 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.980915 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.981040 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.981158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.981275 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.981400 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.981519 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.981636 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] federation.caching = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.981752 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.981876 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.981995 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.caching = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.982120 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.982238 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.982356 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.982479 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.982597 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.982713 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.982838 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.982968 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.983087 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.983204 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.983321 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.983438 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.983629 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.983749 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.983867 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.983984 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.984102 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.984220 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.984346 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.984464 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.984596 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.caching = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.984715 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.984833 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.984951 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.985080 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.985199 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.985316 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.985434 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.985555 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.985678 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.985808 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.985926 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.986043 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.986161 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.986278 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.986396 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.986521 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.986640 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.986757 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.986879 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.986997 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.987115 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.987242 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.987361 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.987479 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.987597 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.987714 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.987831 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.987956 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.988074 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.988198 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.988316 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.988316 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.988582 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.988717 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.988837 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.988954 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.989071 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.989194 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.989314 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.989439 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.989557 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.989674 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.989792 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.989908 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.990026 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.990151 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.990269 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.990387 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.990504 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.990622 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.990747 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.990865 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.990982 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.991099 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.991216 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.991340 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.991466 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.991584 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.991707 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.991825 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.991942 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.992059 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.992188 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.992307 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.992426 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.992560 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.992680 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.992798 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.992927 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.993065 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.993197 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.993197 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.993402 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.993402 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.993631 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.993751 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.993876 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.993994 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.994111 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.994228 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.994353 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.994470 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.994588 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.994706 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.994823 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.994939 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.995064 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.995183 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.995301 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.995418 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.995540 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.995657 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.996291 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.996409 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.996571 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.996692 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.996810 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.996927 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.997057 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.997175 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.997292 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.997410 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.997526 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.997644 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.997769 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.997887 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.998004 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.998122 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.998238 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.998355 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.998480 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.998599 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.998717 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.998834 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.998834 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.999052 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.999179 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.999297 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.999414 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.999530 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.999647 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.999765 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:02.999890 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.000008 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.000126 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.000243 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.000360 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.000477 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.000624 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.000744 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.000862 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.000979 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.001096 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.001212 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.001337 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.001456 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.001572 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.001689 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.001805 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.001922 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.002048 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.002166 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.002283 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.002400 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.002517 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.002633 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.002758 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.002878 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.002995 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.003111 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.003228 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.003344 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.003470 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] resource.caching = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.003588 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.003705 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.003821 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.003940 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.004056 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.004181 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.004299 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.004416 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.004587 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.004707 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.004825 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.004954 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.005075 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.005192 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.005309 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.005426 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.005549 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.005675 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.005792 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.005910 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] role.caching = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.006027 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.006145 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] role.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.006270 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.006389 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.006513 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] resource.caching = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.006632 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.006749 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.006866 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.007058 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.007180 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.007297 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.007414 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.007530 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.007647 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.007773 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.007891 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.008008 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.008127 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.008244 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.008416 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.008569 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.008691 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.008809 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.008927 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.009045 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] role.caching = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.009163 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.009163 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.009394 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] role.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.009511 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.009629 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.009629 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.009848 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.009973 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.010092 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.010209 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.010327 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.010444 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.010560 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.010686 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.010804 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.010921 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.011038 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.011155 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.011273 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.011477 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.012170 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.012292 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.012411 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.012574 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.012695 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.012826 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.012947 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.013064 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.013182 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.013182 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.013411 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.013529 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.013646 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.013763 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.013879 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.013997 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.014122 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.014241 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.014358 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.014476 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.014592 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.014709 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.014709 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] token.caching = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.014939 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.015056 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.015173 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.015290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.015407 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.015537 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.015655 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.015772 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.015889 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.016005 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.016122 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.016248 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.016366 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.016483 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.016618 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.016746 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.016866 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.016991 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.017109 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.017227 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.017344 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.017461 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.017579 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.017705 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.017823 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.017947 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.018065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.018183 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.018300 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] token.caching = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.018426 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.018554 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.018682 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.018801 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.018908 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.019024 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.019142 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.connection = **** {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.019272 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.019391 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.019508 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.019625 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.019744 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.019862 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.019987 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.020106 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.020224 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.020341 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.020459 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.020624 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.020757 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.020877 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.020993 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.021110 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.021227 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.021344 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.021469 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.021587 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.021705 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.021824 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.021940 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.022067 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.022193 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.022312 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.022438 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.022557 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.022674 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.connection = **** {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.022792 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.022916 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.023034 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.023160 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.023279 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.023279 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.023499 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.023616 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 27 09:23:03.023734 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.023734 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.023734 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.023734 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.023734 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.023734 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.023734 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.023734 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.023734 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.024702 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.024702 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.024702 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.024702 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.024702 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.024702 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 09:23:03.024702 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 27 09:23:03.047891 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.048120 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.048345 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.050562 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.050787 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.051018 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.056597 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.059887 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.085243 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.085469 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.086110 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.086343 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.138509 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.139312 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.187120 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.187350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.187976 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.188211 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.341421 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.341652 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.341746 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.341825 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.341925 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.342004 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.342079 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.342183 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.342268 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.342398 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.342483 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.342610 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.365861 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.366080 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.366178 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.366291 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.366370 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.366486 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.366571 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.366707 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.366801 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.366920 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.367004 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.367129 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.367212 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.367337 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.378743 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.378910 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.391898 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.392042 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.392179 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.392273 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.392351 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.392426 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.392500 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.392591 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.392743 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.392829 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.392961 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.393045 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.393175 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.393268 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 09:23:03.410316 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.410525 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 27 09:23:03.410753 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.410864 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.410948 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 27 09:23:03.411062 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 27 09:23:03.411293 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.411376 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.411496 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 27 09:23:03.411581 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.411772 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.411855 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.412039 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.412120 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.412302 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.412593 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.422107 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.422384 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.422651 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.422733 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.422929 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.423025 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.423205 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.423493 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.429865 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 09:23:03.430446 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 09:23:03.432762 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 09:23:03.433047 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.433919 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.434006 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 09:23:03.434203 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.434284 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.434486 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.434570 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.434748 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.434831 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.435015 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.435106 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.435284 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.435367 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.435551 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.435632 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.435821 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.435903 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.436088 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.436169 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.436359 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.436657 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.458526 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.459199 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.459966 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 09:23:03.460248 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.460684 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 09:23:03.460973 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.465422 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 09:23:03.465705 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.465986 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.466143 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 09:23:03.466432 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.466719 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.472611 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 09:23:03.472897 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.473314 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 09:23:03.473604 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.478463 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 09:23:03.478757 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.479054 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.479270 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 09:23:03.479354 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.479571 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.479655 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.479865 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.479948 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.480752 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.480859 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.481042 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.481125 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.481325 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.481613 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.481903 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.496569 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.496849 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.497554 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.497837 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.500706 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 09:23:03.501720 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 09:23:03.503905 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.504202 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.504957 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.505270 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.508419 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 09:23:03.509492 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 09:23:03.511578 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 09:23:03.512717 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 09:23:03.514807 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.515112 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.515398 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.515698 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.515936 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.516018 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.516241 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.516326 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.517493 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.517493 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.517493 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.517493 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.532438 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 09:23:03.532750 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.533018 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.533278 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.533614 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 09:23:03.533908 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.534181 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.534449 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.540060 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.541317 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.541624 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.543405 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.543751 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.544036 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.544980 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.545272 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.546712 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 09:23:03.546995 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.547275 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.547558 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.547932 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 09:23:03.548022 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.548222 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.548534 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.548825 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.549217 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.558992 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 09:23:03.559276 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.559557 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.559834 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.560227 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.560320 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 09:23:03.560503 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.560605 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.560822 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.560906 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.561167 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.561566 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.561852 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.562136 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.576028 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 09:23:03.577458 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 09:23:03.578935 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 09:23:03.580358 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 09:23:03.581802 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.583227 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.583322 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.584715 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.584798 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 09:23:03.585006 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.585415 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.586307 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 09:23:03.586604 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.587009 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.592316 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.593811 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 09:23:03.593935 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.595406 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 09:23:03.596964 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.597360 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.597642 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.597914 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.598561 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.598968 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.599258 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.599540 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.606265 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 09:23:03.606551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.606827 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.607859 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 09:23:03.608158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.608439 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.613657 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 09:23:03.613940 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.614216 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.614488 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.614756 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.615038 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.615310 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.615401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 09:23:03.615589 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.615672 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.615856 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.615938 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.616129 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.616220 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.616405 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.616487 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.616708 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.616793 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.616977 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.617058 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.617241 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.617322 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.617527 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.617801 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.618090 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.618364 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.618631 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.618934 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 09:23:03.650721 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x561472b829c0 pid: 81437 (default app) Oct 27 09:23:03.653265 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x561472b829c0 pid: 81436 (default app) Oct 27 09:23:03.654517 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5d4da-5c38-472a-b1e2-c99ffef0a8a3 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:03.654800 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5d4da-5c38-472a-b1e2-c99ffef0a8a3 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:03.655069 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5d4da-5c38-472a-b1e2-c99ffef0a8a3 None None] PATH_INFO: `/v3` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:03.659216 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1/1] 10.222.0.70 () {50 vars in 791 bytes} [Thu Oct 27 09:23:03 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:10.405501 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3e274-8ffb-4761-a03f-520777aade57 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:10.406000 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3e274-8ffb-4761-a03f-520777aade57 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:10.406106 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3e274-8ffb-4761-a03f-520777aade57 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:10.410309 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1/2] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:23:10 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 27 09:23:10.415554 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be8e5a-18ef-4a26-be46-d9518f059192 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:10.416033 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be8e5a-18ef-4a26-be46-d9518f059192 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:10.416125 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be8e5a-18ef-4a26-be46-d9518f059192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:10.485295 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: INFO dbcounter [None req-16be8e5a-18ef-4a26-be46-d9518f059192 None None] Registered counter for database keystone Oct 27 09:23:10.503721 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG oslo_db.sqlalchemy.engines [None req-16be8e5a-18ef-4a26-be46-d9518f059192 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81437) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 27 09:23:10.509827 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writer thread running {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 27 09:23:10.688263 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG passlib.handlers.bcrypt [None req-16be8e5a-18ef-4a26-be46-d9518f059192 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81437) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 27 09:23:10.688681 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG passlib.handlers.bcrypt [None req-16be8e5a-18ef-4a26-be46-d9518f059192 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81437) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 27 09:23:10.746320 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-16be8e5a-18ef-4a26-be46-d9518f059192 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:23:10.747914 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-16be8e5a-18ef-4a26-be46-d9518f059192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:23:10.872546 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 2/3] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:23:10 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 458 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 27 09:23:11.311204 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3e274-8ffb-4761-a03f-520777aade57 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:11.379809 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: INFO dbcounter [None req-07e3e274-8ffb-4761-a03f-520777aade57 None None] Registered counter for database keystone Oct 27 09:23:11.401386 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG oslo_db.sqlalchemy.engines [None req-07e3e274-8ffb-4761-a03f-520777aade57 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81436) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 27 09:23:11.407408 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writer thread running {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 27 09:23:11.526704 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3e274-8ffb-4761-a03f-520777aade57 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:11.530908 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963a19bb-e927-45ac-8d33-a7c6dc488d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:11.531514 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a19bb-e927-45ac-8d33-a7c6dc488d28 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:11.531787 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a19bb-e927-45ac-8d33-a7c6dc488d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:11.532050 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a19bb-e927-45ac-8d33-a7c6dc488d28 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:11.540145 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963a19bb-e927-45ac-8d33-a7c6dc488d28 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:11.592059 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: WARNING keystone.common.rbac_enforcer.enforcer [None req-963a19bb-e927-45ac-8d33-a7c6dc488d28 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 27 09:23:11.592400 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963a19bb-e927-45ac-8d33-a7c6dc488d28 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:11.602147 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 2/4] 10.222.0.70 () {58 vars in 1324 bytes} [Thu Oct 27 09:23:11 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 27 09:23:11.608062 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16be8e5a-18ef-4a26-be46-d9518f059192 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:11.625217 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16be8e5a-18ef-4a26-be46-d9518f059192 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:11.629668 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d4b26-71ad-4688-ac6a-12237f18c43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:11.630769 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d4b26-71ad-4688-ac6a-12237f18c43e admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:11.631047 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d4b26-71ad-4688-ac6a-12237f18c43e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:11.631313 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d4b26-71ad-4688-ac6a-12237f18c43e admin admin] PATH_INFO: `/v3/projects/d2f8e01e4d9d43978ddeb5033ead57f5` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:11.632936 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7d4b26-71ad-4688-ac6a-12237f18c43e admin admin] RBAC: Authorizing `identity:get_project(project_id=d2f8e01e4d9d43978ddeb5033ead57f5)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:11.686742 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: WARNING keystone.common.rbac_enforcer.enforcer [None req-7c7d4b26-71ad-4688-ac6a-12237f18c43e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 27 09:23:11.687089 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7d4b26-71ad-4688-ac6a-12237f18c43e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:11.691621 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 3/5] 10.222.0.70 () {56 vars in 1207 bytes} [Thu Oct 27 09:23:11 2022] GET /identity/v3/projects/d2f8e01e4d9d43978ddeb5033ead57f5 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:13.825207 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba6e4b-78f4-4e13-8642-6985411d3938 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:13.825730 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba6e4b-78f4-4e13-8642-6985411d3938 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:13.825730 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba6e4b-78f4-4e13-8642-6985411d3938 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:13.829861 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 3/6] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:23:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:23:13.835485 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d08565-95c9-4b28-a1ee-a54c0bee2d72 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:13.835842 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d08565-95c9-4b28-a1ee-a54c0bee2d72 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:13.836026 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d08565-95c9-4b28-a1ee-a54c0bee2d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:13.855884 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a6d08565-95c9-4b28-a1ee-a54c0bee2d72 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:23:13.857439 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a6d08565-95c9-4b28-a1ee-a54c0bee2d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:23:13.865237 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 4/7] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:23:13 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 27 09:23:14.138431 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ba6e4b-78f4-4e13-8642-6985411d3938 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:14.150492 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ba6e4b-78f4-4e13-8642-6985411d3938 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:14.154755 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df8ce5-52cb-46bb-a0f8-67779aba132c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:14.155352 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df8ce5-52cb-46bb-a0f8-67779aba132c admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:14.155628 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df8ce5-52cb-46bb-a0f8-67779aba132c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:14.155901 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df8ce5-52cb-46bb-a0f8-67779aba132c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:14.163491 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56df8ce5-52cb-46bb-a0f8-67779aba132c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:14.165307 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56df8ce5-52cb-46bb-a0f8-67779aba132c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:14.174495 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 4/8] 10.222.0.70 () {58 vars in 1324 bytes} [Thu Oct 27 09:23:14 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 27 09:23:14.179881 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d08565-95c9-4b28-a1ee-a54c0bee2d72 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:14.189513 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d08565-95c9-4b28-a1ee-a54c0bee2d72 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:14.193803 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37eba6b-97db-4689-a8f0-441ebd3f20c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:14.194430 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37eba6b-97db-4689-a8f0-441ebd3f20c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:14.194710 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37eba6b-97db-4689-a8f0-441ebd3f20c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:14.194982 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37eba6b-97db-4689-a8f0-441ebd3f20c2 admin admin] PATH_INFO: `/v3/users/4e3d5d12b2ba49459e93d0a715783a8a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:14.196274 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37eba6b-97db-4689-a8f0-441ebd3f20c2 admin admin] RBAC: Authorizing `identity:get_user(user_id=4e3d5d12b2ba49459e93d0a715783a8a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:14.198131 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37eba6b-97db-4689-a8f0-441ebd3f20c2 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:14.199670 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 5/9] 10.222.0.70 () {56 vars in 1198 bytes} [Thu Oct 27 09:23:14 2022] GET /identity/v3/users/4e3d5d12b2ba49459e93d0a715783a8a => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:16.390240 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99529c3a-b20d-4b16-9587-c0af06d72b86 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:16.390792 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99529c3a-b20d-4b16-9587-c0af06d72b86 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:16.390792 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99529c3a-b20d-4b16-9587-c0af06d72b86 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:16.392106 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 5/10] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:23:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:23:16.397202 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a045113-a97d-4f97-afff-002d11bed800 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:16.397611 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a045113-a97d-4f97-afff-002d11bed800 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:16.397818 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a045113-a97d-4f97-afff-002d11bed800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:16.416125 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-2a045113-a97d-4f97-afff-002d11bed800 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:23:16.417329 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2a045113-a97d-4f97-afff-002d11bed800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:23:16.436104 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa4662-ec87-4b93-8567-90b94b45b359 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:16.436393 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa4662-ec87-4b93-8567-90b94b45b359 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:16.436706 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa4662-ec87-4b93-8567-90b94b45b359 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:16.438011 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 6/11] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:23:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 27 09:23:16.443024 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d10abc-61ae-4730-ba4b-226f82f46c76 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:16.443306 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d10abc-61ae-4730-ba4b-226f82f46c76 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:16.443576 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d10abc-61ae-4730-ba4b-226f82f46c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:16.486165 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG passlib.handlers.bcrypt [None req-f7d10abc-61ae-4730-ba4b-226f82f46c76 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81436) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 27 09:23:16.486568 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG passlib.handlers.bcrypt [None req-f7d10abc-61ae-4730-ba4b-226f82f46c76 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81436) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 27 09:23:16.498352 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 6/12] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:23:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:23:16.537064 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f7d10abc-61ae-4730-ba4b-226f82f46c76 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:23:16.538563 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f7d10abc-61ae-4730-ba4b-226f82f46c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:23:16.636464 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 7/13] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:23:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 27 09:23:16.773348 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a045113-a97d-4f97-afff-002d11bed800 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:16.798100 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:16.882134 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a045113-a97d-4f97-afff-002d11bed800 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:16.902304 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d10abc-61ae-4730-ba4b-226f82f46c76 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:16.917265 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8321f8c7-01f2-4beb-b201-c6837b82a366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:16.917899 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8321f8c7-01f2-4beb-b201-c6837b82a366 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:16.918180 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8321f8c7-01f2-4beb-b201-c6837b82a366 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:16.918456 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8321f8c7-01f2-4beb-b201-c6837b82a366 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:16.929430 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8321f8c7-01f2-4beb-b201-c6837b82a366 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:23:16.938764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-8321f8c7-01f2-4beb-b201-c6837b82a366 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 09:23:16.938764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:23:16.938764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:23:16.938764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:23:16.938764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:23:16.938764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:23:16.938764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:23:16.938764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:23:16.938764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:23:16.938764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:23:16.938764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:23:16.938764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:23:16.938764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:23:16.938764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:23:16.938764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:23:16.938764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:23:16.939697 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:23:16.939697 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:23:16.939697 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:23:16.939697 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:23:16.939697 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:23:16.939697 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:23:16.939697 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:23:16.939697 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:23:16.939697 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:23:16.939697 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:23:16.939697 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:23:16.939697 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:23:16.939697 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:23:16.939697 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:23:16.939697 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:23:16.939697 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:23:16.940623 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:23:16.940623 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:23:16.940623 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:23:16.940623 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:23:16.940623 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:23:16.940623 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:23:16.940623 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:23:16.940623 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:23:16.940623 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:23:16.940623 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:23:16.940623 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 09:23:16.940623 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:23:16.940623 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 7/14] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Oct 27 09:23:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:23:16.947463 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8321f8c7-01f2-4beb-b201-c6837b82a366 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:17.004481 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d10abc-61ae-4730-ba4b-226f82f46c76 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:17.006799 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:17.031386 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:17.045454 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb20075-a674-4444-ada7-a6b960888828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:17.046320 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb20075-a674-4444-ada7-a6b960888828 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:17.046600 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb20075-a674-4444-ada7-a6b960888828 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:17.046871 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb20075-a674-4444-ada7-a6b960888828 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:17.047844 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb20075-a674-4444-ada7-a6b960888828 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:17.051036 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8321f8c7-01f2-4beb-b201-c6837b82a366 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:17.052535 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb20075-a674-4444-ada7-a6b960888828 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:17.064371 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.sql.core [None req-deb20075-a674-4444-ada7-a6b960888828 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 27 09:23:17.064371 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 27 09:23:17.064371 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [parameters: {'id': '4d05f96419404c4287d90ee46d5f11e8', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 27 09:23:17.064371 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81436) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 27 09:23:17.082990 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-deb20075-a674-4444-ada7-a6b960888828 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 27 09:23:17.082990 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:23:17.082990 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 27 09:23:17.082990 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 27 09:23:17.082990 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 27 09:23:17.082990 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 27 09:23:17.082990 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 27 09:23:17.082990 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application result = self._query(query) Oct 27 09:23:17.082990 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 27 09:23:17.082990 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application conn.query(q) Oct 27 09:23:17.082990 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 27 09:23:17.082990 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 27 09:23:17.082990 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 27 09:23:17.082990 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application result.read() Oct 27 09:23:17.084088 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 27 09:23:17.084088 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 27 09:23:17.084088 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 27 09:23:17.084088 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 27 09:23:17.084088 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 27 09:23:17.084088 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 27 09:23:17.084088 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 27 09:23:17.084088 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 27 09:23:17.084088 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 27 09:23:17.084088 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:23:17.084088 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 27 09:23:17.084088 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:23:17.084088 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:23:17.084088 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 27 09:23:17.084088 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 27 09:23:17.084088 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 27 09:23:17.084088 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 27 09:23:17.084088 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 27 09:23:17.089561 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application next(self.gen) Oct 27 09:23:17.089561 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 27 09:23:17.089561 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application yield resource Oct 27 09:23:17.089561 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 27 09:23:17.089561 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application next(self.gen) Oct 27 09:23:17.089561 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 27 09:23:17.089561 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self.session.rollback() Oct 27 09:23:17.089561 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 27 09:23:17.089561 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self.force_reraise() Oct 27 09:23:17.089561 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 27 09:23:17.089561 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise self.value Oct 27 09:23:17.089561 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 27 09:23:17.089561 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 27 09:23:17.089561 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 27 09:23:17.089561 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application session.commit() Oct 27 09:23:17.089561 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 27 09:23:17.089561 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 27 09:23:17.090685 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 27 09:23:17.090685 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._prepare_impl() Oct 27 09:23:17.090685 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 27 09:23:17.090685 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self.session.flush() Oct 27 09:23:17.090685 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 27 09:23:17.090685 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._flush(objects) Oct 27 09:23:17.090685 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 27 09:23:17.090685 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 27 09:23:17.090685 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 27 09:23:17.090685 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application compat.raise_( Oct 27 09:23:17.090685 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 27 09:23:17.090685 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception Oct 27 09:23:17.090685 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 27 09:23:17.090685 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application flush_context.execute() Oct 27 09:23:17.090685 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 27 09:23:17.090685 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rec.execute(self) Oct 27 09:23:17.090685 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 27 09:23:17.091758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 27 09:23:17.091758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 27 09:23:17.091758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 27 09:23:17.091758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 27 09:23:17.091758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 27 09:23:17.091758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 27 09:23:17.091758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 27 09:23:17.091758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 27 09:23:17.091758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 27 09:23:17.091758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 27 09:23:17.091758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 27 09:23:17.091758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 27 09:23:17.091758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 27 09:23:17.091758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 27 09:23:17.091758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 27 09:23:17.092732 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 27 09:23:17.092732 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception Oct 27 09:23:17.092732 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 27 09:23:17.092732 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 27 09:23:17.092732 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 27 09:23:17.092732 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 27 09:23:17.092732 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 27 09:23:17.092732 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application result = self._query(query) Oct 27 09:23:17.092732 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 27 09:23:17.092732 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application conn.query(q) Oct 27 09:23:17.092732 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 27 09:23:17.092732 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 27 09:23:17.092732 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 27 09:23:17.092732 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application result.read() Oct 27 09:23:17.092732 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 27 09:23:17.092732 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 27 09:23:17.092732 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 27 09:23:17.093840 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 27 09:23:17.093840 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 27 09:23:17.093840 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 27 09:23:17.093840 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 27 09:23:17.093840 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 27 09:23:17.093840 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 27 09:23:17.093840 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 27 09:23:17.093840 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [parameters: {'id': '4d05f96419404c4287d90ee46d5f11e8', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 27 09:23:17.093840 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 27 09:23:17.093840 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:23:17.093840 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 09:23:17.093840 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:23:17.093840 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:23:17.093840 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 27 09:23:17.093840 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 27 09:23:17.095040 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 27 09:23:17.095040 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 27 09:23:17.095040 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 27 09:23:17.095040 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:23:17.095040 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 09:23:17.095040 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:23:17.095040 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:23:17.095040 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:23:17.095040 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:23:17.095040 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:23:17.095040 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:23:17.095040 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:23:17.095040 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:23:17.095040 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:23:17.095040 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:23:17.095040 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:23:17.095040 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:23:17.096601 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 27 09:23:17.096601 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 27 09:23:17.096601 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:23:17.096601 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:23:17.096601 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 27 09:23:17.096601 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 27 09:23:17.096601 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:23:17.096601 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:23:17.096601 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 27 09:23:17.096601 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 27 09:23:17.096601 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 27 09:23:17.096601 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:23:17.096601 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 8/15] 10.222.0.70 () {60 vars in 1137 bytes} [Thu Oct 27 09:23:16 2022] POST /identity/v3/domains => generated 190 bytes in 182 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 27 09:23:17.096601 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb20075-a674-4444-ada7-a6b960888828 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:17.097489 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f3163e-3ccf-44df-a7c4-43c0424b8c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:17.097489 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3163e-3ccf-44df-a7c4-43c0424b8c33 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:17.097489 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3163e-3ccf-44df-a7c4-43c0424b8c33 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:17.097489 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3163e-3ccf-44df-a7c4-43c0424b8c33 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:17.097834 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f3163e-3ccf-44df-a7c4-43c0424b8c33 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:23:17.098140 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f3163e-3ccf-44df-a7c4-43c0424b8c33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:17.099551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f3163e-3ccf-44df-a7c4-43c0424b8c33 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:17.112609 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 8/16] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Oct 27 09:23:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:23:17.118239 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f3163e-3ccf-44df-a7c4-43c0424b8c33 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:17.194855 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb20075-a674-4444-ada7-a6b960888828 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:17.222881 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f3163e-3ccf-44df-a7c4-43c0424b8c33 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:17.232635 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11779c88-f56a-4c88-9f95-4eb24aa02aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:17.233489 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11779c88-f56a-4c88-9f95-4eb24aa02aaa None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:17.233767 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11779c88-f56a-4c88-9f95-4eb24aa02aaa None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:17.234036 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11779c88-f56a-4c88-9f95-4eb24aa02aaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:17.255073 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11779c88-f56a-4c88-9f95-4eb24aa02aaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:17.256913 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11779c88-f56a-4c88-9f95-4eb24aa02aaa None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:17.257038 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2414dd09-6c35-4a84-af76-2ae9b693b82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:17.257634 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414dd09-6c35-4a84-af76-2ae9b693b82d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:17.257914 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414dd09-6c35-4a84-af76-2ae9b693b82d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:17.258185 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414dd09-6c35-4a84-af76-2ae9b693b82d None admin] PATH_INFO: `/v3/users/4e3d5d12b2ba49459e93d0a715783a8a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:17.259324 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2414dd09-6c35-4a84-af76-2ae9b693b82d None admin] RBAC: Authorizing `identity:get_user(user_id=4e3d5d12b2ba49459e93d0a715783a8a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:17.260814 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2414dd09-6c35-4a84-af76-2ae9b693b82d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:17.262370 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 9/17] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Oct 27 09:23:17 2022] GET /identity/v3/users/4e3d5d12b2ba49459e93d0a715783a8a => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:23:17.267834 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2414dd09-6c35-4a84-af76-2ae9b693b82d None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:17.275066 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-11779c88-f56a-4c88-9f95-4eb24aa02aaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:23:17.275066 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:23:17.275066 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:23:17.275066 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:23:17.275066 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:23:17.275066 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:23:17.275066 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:23:17.275066 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:23:17.275066 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:23:17.275066 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:23:17.275066 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:23:17.275066 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:23:17.275066 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:23:17.275066 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:23:17.275066 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:23:17.275066 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:23:17.276158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:23:17.276158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:23:17.276158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:23:17.276158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:23:17.276158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:23:17.276158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:23:17.276158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:23:17.276158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:23:17.276158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:23:17.276158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:23:17.276158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:23:17.276158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:23:17.276158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:23:17.276158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:23:17.276158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:23:17.276158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:23:17.276158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:23:17.277247 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:23:17.277247 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:23:17.277247 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:23:17.277247 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 9/18] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:23:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:23:17.282087 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11779c88-f56a-4c88-9f95-4eb24aa02aaa None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:17.288000 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:17.306832 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:17.384059 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2414dd09-6c35-4a84-af76-2ae9b693b82d None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:17.388927 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11779c88-f56a-4c88-9f95-4eb24aa02aaa None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:17.420045 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fcabaf-0bce-4a89-bd99-a57c2ebf0594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:17.420714 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fcabaf-0bce-4a89-bd99-a57c2ebf0594 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:17.421004 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fcabaf-0bce-4a89-bd99-a57c2ebf0594 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:17.421280 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fcabaf-0bce-4a89-bd99-a57c2ebf0594 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:17.422405 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fcabaf-0bce-4a89-bd99-a57c2ebf0594 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:17.423870 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fcabaf-0bce-4a89-bd99-a57c2ebf0594 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:17.425452 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 10/19] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:23:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:23:17.427265 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb373c3-4920-4890-9c54-d8c2b9517758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:17.427875 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb373c3-4920-4890-9c54-d8c2b9517758 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:17.428153 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb373c3-4920-4890-9c54-d8c2b9517758 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:17.428424 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb373c3-4920-4890-9c54-d8c2b9517758 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:17.429091 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb373c3-4920-4890-9c54-d8c2b9517758 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:23:17.429388 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb373c3-4920-4890-9c54-d8c2b9517758 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:17.430759 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb373c3-4920-4890-9c54-d8c2b9517758 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:17.431062 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fcabaf-0bce-4a89-bd99-a57c2ebf0594 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:17.447913 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 10/20] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:23:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:23:17.519359 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:17.545616 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fcabaf-0bce-4a89-bd99-a57c2ebf0594 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:17.581347 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068fb24d-a224-4bc7-9b60-41dcf2770b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:17.582028 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fb24d-a224-4bc7-9b60-41dcf2770b96 None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:17.582307 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fb24d-a224-4bc7-9b60-41dcf2770b96 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:17.582586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fb24d-a224-4bc7-9b60-41dcf2770b96 None admin] PATH_INFO: `/v3/domains/default/users/4e3d5d12b2ba49459e93d0a715783a8a/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:17.584691 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068fb24d-a224-4bc7-9b60-41dcf2770b96 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=4e3d5d12b2ba49459e93d0a715783a8a, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:17.586180 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068fb24d-a224-4bc7-9b60-41dcf2770b96 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:17.702793 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 11/21] 10.222.0.70 () {58 vars in 1361 bytes} [Thu Oct 27 09:23:17 2022] PUT /identity/v3/domains/default/users/4e3d5d12b2ba49459e93d0a715783a8a/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:23:20.070634 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a425a-b493-438c-969f-6d9cd6f9fd43 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:20.071149 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a425a-b493-438c-969f-6d9cd6f9fd43 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:20.071223 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a425a-b493-438c-969f-6d9cd6f9fd43 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:20.072499 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 11/22] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:23:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:23:20.084531 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77745f6-1ca6-4eff-ac58-4b564e562128 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:20.085437 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77745f6-1ca6-4eff-ac58-4b564e562128 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:20.085951 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77745f6-1ca6-4eff-ac58-4b564e562128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:20.113604 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-e77745f6-1ca6-4eff-ac58-4b564e562128 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:23:20.115450 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e77745f6-1ca6-4eff-ac58-4b564e562128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:23:20.178848 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b08c4-5328-4f59-98e3-37d453be22ff None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:20.179162 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b08c4-5328-4f59-98e3-37d453be22ff None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:20.179397 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b08c4-5328-4f59-98e3-37d453be22ff None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:20.181195 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 12/23] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:23:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 27 09:23:20.187206 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd59cf5-b6ea-4fe4-b8b2-e5467380d585 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:20.187493 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd59cf5-b6ea-4fe4-b8b2-e5467380d585 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:20.187778 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd59cf5-b6ea-4fe4-b8b2-e5467380d585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:20.212996 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 12/24] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:23:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:23:20.217604 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-cdd59cf5-b6ea-4fe4-b8b2-e5467380d585 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:23:20.218855 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-cdd59cf5-b6ea-4fe4-b8b2-e5467380d585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:23:20.322545 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 13/25] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:23:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 27 09:23:20.343309 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a6c4b-16f8-44a6-a637-6464887b830a None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:20.343885 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a6c4b-16f8-44a6-a637-6464887b830a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:20.344366 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a6c4b-16f8-44a6-a637-6464887b830a None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:20.349392 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 13/26] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:23:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:23:20.355212 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f5a78-4b73-4a54-92f4-ab33c4f29cf2 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:20.355501 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f5a78-4b73-4a54-92f4-ab33c4f29cf2 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:20.355775 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f5a78-4b73-4a54-92f4-ab33c4f29cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:20.378415 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-016f5a78-4b73-4a54-92f4-ab33c4f29cf2 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:23:20.379614 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-016f5a78-4b73-4a54-92f4-ab33c4f29cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:23:20.382596 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:20.472485 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 14/27] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:23:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:23:20.657298 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a6c4b-16f8-44a6-a637-6464887b830a None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:20.658209 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016f5a78-4b73-4a54-92f4-ab33c4f29cf2 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:20.810258 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016f5a78-4b73-4a54-92f4-ab33c4f29cf2 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:20.810652 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:20.854833 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7026b-2a56-4846-a1e0-dfa920b80a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:20.855465 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7026b-2a56-4846-a1e0-dfa920b80a14 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:20.855742 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7026b-2a56-4846-a1e0-dfa920b80a14 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:20.856009 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7026b-2a56-4846-a1e0-dfa920b80a14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:20.868540 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:20.869071 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f7026b-2a56-4846-a1e0-dfa920b80a14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:20.871217 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f7026b-2a56-4846-a1e0-dfa920b80a14 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:20.890888 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a6c4b-16f8-44a6-a637-6464887b830a None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:20.898483 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-90f7026b-2a56-4846-a1e0-dfa920b80a14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:23:20.898483 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:23:20.898483 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:23:20.898483 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:23:20.898483 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:23:20.898483 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:23:20.898483 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:23:20.898483 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:23:20.898483 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:23:20.898483 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:23:20.898483 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:23:20.898483 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:23:20.898483 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:23:20.898483 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:23:20.898483 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:23:20.898483 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:23:20.899364 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:23:20.899364 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:23:20.899364 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:23:20.899364 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:23:20.899364 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:23:20.899364 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:23:20.899364 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:23:20.899364 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:23:20.899364 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:23:20.899364 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:23:20.899364 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:23:20.899364 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:23:20.899364 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:23:20.899364 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:23:20.899364 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:23:20.899364 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:23:20.899364 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:23:20.900185 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:23:20.900185 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:23:20.900185 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:23:20.904196 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 15/28] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:23:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 09:23:20.905040 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7026b-2a56-4846-a1e0-dfa920b80a14 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:20.965978 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ff2ec-9d5d-4350-8c58-69e9d3329072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:20.966527 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ff2ec-9d5d-4350-8c58-69e9d3329072 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:20.966867 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ff2ec-9d5d-4350-8c58-69e9d3329072 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:20.967249 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ff2ec-9d5d-4350-8c58-69e9d3329072 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:20.968042 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823ff2ec-9d5d-4350-8c58-69e9d3329072 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:20.969605 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823ff2ec-9d5d-4350-8c58-69e9d3329072 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:21.009441 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 14/29] 10.222.0.70 () {60 vars in 1131 bytes} [Thu Oct 27 09:23:20 2022] POST /identity/v3/roles => generated 219 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:23:21.010486 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ff2ec-9d5d-4350-8c58-69e9d3329072 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:21.019212 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7026b-2a56-4846-a1e0-dfa920b80a14 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:21.058933 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e819ccb4-f692-4385-a444-480798f801d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:21.059581 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819ccb4-f692-4385-a444-480798f801d9 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:21.059858 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819ccb4-f692-4385-a444-480798f801d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:21.060121 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819ccb4-f692-4385-a444-480798f801d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:21.072334 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e819ccb4-f692-4385-a444-480798f801d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:23:21.080797 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-e819ccb4-f692-4385-a444-480798f801d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 09:23:21.080797 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:23:21.080797 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:23:21.080797 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:23:21.080797 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:23:21.080797 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:23:21.080797 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:23:21.080797 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:23:21.080797 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:23:21.080797 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:23:21.080797 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:23:21.080797 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:23:21.080797 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:23:21.080797 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:23:21.080797 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:23:21.080797 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:23:21.083070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:23:21.083070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:23:21.083070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:23:21.083070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:23:21.083070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:23:21.083070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:23:21.083070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:23:21.083070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:23:21.083070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:23:21.083070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:23:21.083070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:23:21.083070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:23:21.083070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:23:21.083070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:23:21.083070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:23:21.083070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:23:21.084738 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:23:21.084738 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:23:21.084738 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:23:21.084738 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:23:21.084738 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:23:21.084738 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:23:21.084738 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:23:21.084738 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:23:21.084738 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:23:21.084738 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:23:21.084738 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 09:23:21.084738 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:23:21.084738 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 16/30] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Oct 27 09:23:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:23:21.094826 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e819ccb4-f692-4385-a444-480798f801d9 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:21.226878 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ff2ec-9d5d-4350-8c58-69e9d3329072 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:21.234336 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:21.292484 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:21.306913 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ff2c6a-7750-441d-9120-cd428244057e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:21.307575 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff2c6a-7750-441d-9120-cd428244057e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:21.307854 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff2c6a-7750-441d-9120-cd428244057e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:21.308126 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff2c6a-7750-441d-9120-cd428244057e None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:21.308814 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ff2c6a-7750-441d-9120-cd428244057e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:23:21.309121 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ff2c6a-7750-441d-9120-cd428244057e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:21.310524 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ff2c6a-7750-441d-9120-cd428244057e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:21.326307 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e819ccb4-f692-4385-a444-480798f801d9 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:21.331175 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 15/31] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:23:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:21.332461 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c1ce7-53bd-498f-9d3f-b426126c07d3 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:21.332778 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c1ce7-53bd-498f-9d3f-b426126c07d3 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:21.333150 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c1ce7-53bd-498f-9d3f-b426126c07d3 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:21.339033 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 16/32] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:23:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:23:21.339800 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c1ce7-53bd-498f-9d3f-b426126c07d3 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:21.370541 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c719ee7-1768-44d9-92d5-0e9d0826d479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:21.371555 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c719ee7-1768-44d9-92d5-0e9d0826d479 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:21.372089 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c719ee7-1768-44d9-92d5-0e9d0826d479 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:21.372587 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c719ee7-1768-44d9-92d5-0e9d0826d479 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:21.373704 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c719ee7-1768-44d9-92d5-0e9d0826d479 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:23:21.374616 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c719ee7-1768-44d9-92d5-0e9d0826d479 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:21.376595 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c719ee7-1768-44d9-92d5-0e9d0826d479 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:21.393055 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 17/33] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Oct 27 09:23:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:21.395328 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d70631-27a9-470e-9192-5060b3363704 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:21.395797 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d70631-27a9-470e-9192-5060b3363704 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:21.396263 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d70631-27a9-470e-9192-5060b3363704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:21.418382 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f8d70631-27a9-470e-9192-5060b3363704 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:23:21.420233 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f8d70631-27a9-470e-9192-5060b3363704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:23:21.467083 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c1ce7-53bd-498f-9d3f-b426126c07d3 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:21.510026 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 18/34] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:23:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:23:21.511750 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d70631-27a9-470e-9192-5060b3363704 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:21.518613 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51229bc-9737-4c7a-8921-126ef6d736ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:21.519323 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51229bc-9737-4c7a-8921-126ef6d736ce None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:21.519683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51229bc-9737-4c7a-8921-126ef6d736ce None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:21.520035 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51229bc-9737-4c7a-8921-126ef6d736ce None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:21.520850 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51229bc-9737-4c7a-8921-126ef6d736ce None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:21.522382 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51229bc-9737-4c7a-8921-126ef6d736ce None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:21.559375 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 17/35] 10.222.0.70 () {60 vars in 1141 bytes} [Thu Oct 27 09:23:21 2022] POST /identity/v3/projects => generated 301 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:23:21.570980 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:21.615827 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41687e-cbab-41c0-a744-f5add54a37fa None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:21.616170 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41687e-cbab-41c0-a744-f5add54a37fa None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:21.616404 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41687e-cbab-41c0-a744-f5add54a37fa None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:21.618241 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 18/36] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:23:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 27 09:23:21.624935 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbbf34a-6bec-4050-9fd9-e8c335969e76 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:21.625532 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbbf34a-6bec-4050-9fd9-e8c335969e76 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:21.626021 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbbf34a-6bec-4050-9fd9-e8c335969e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:21.648899 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-ccbbf34a-6bec-4050-9fd9-e8c335969e76 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:23:21.651960 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ccbbf34a-6bec-4050-9fd9-e8c335969e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:23:21.663723 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d70631-27a9-470e-9192-5060b3363704 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:21.705035 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:21.736812 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94be6cda-8534-4888-be6f-8a2e967836ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:21.737484 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be6cda-8534-4888-be6f-8a2e967836ce None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:21.737748 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be6cda-8534-4888-be6f-8a2e967836ce None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:21.738013 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be6cda-8534-4888-be6f-8a2e967836ce None admin] PATH_INFO: `/v3/users/4e3d5d12b2ba49459e93d0a715783a8a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:21.739380 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94be6cda-8534-4888-be6f-8a2e967836ce None admin] RBAC: Authorizing `identity:get_user(user_id=4e3d5d12b2ba49459e93d0a715783a8a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:21.745128 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94be6cda-8534-4888-be6f-8a2e967836ce None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:21.747331 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 19/37] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Oct 27 09:23:21 2022] GET /identity/v3/users/4e3d5d12b2ba49459e93d0a715783a8a => generated 247 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:21.750068 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 19/38] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:23:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 27 09:23:21.752909 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94be6cda-8534-4888-be6f-8a2e967836ce None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:21.850175 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbbf34a-6bec-4050-9fd9-e8c335969e76 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:21.865592 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94be6cda-8534-4888-be6f-8a2e967836ce None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:21.882233 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:21.910930 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa1dcf2-7038-451a-9c86-1b063fbedada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:21.911565 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1dcf2-7038-451a-9c86-1b063fbedada None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:21.911842 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1dcf2-7038-451a-9c86-1b063fbedada None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:21.912111 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1dcf2-7038-451a-9c86-1b063fbedada None admin] PATH_INFO: `/v3/domains/default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:21.913457 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa1dcf2-7038-451a-9c86-1b063fbedada None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:21.914938 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa1dcf2-7038-451a-9c86-1b063fbedada None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:21.916944 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 20/39] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:23:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:23:21.923917 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa1dcf2-7038-451a-9c86-1b063fbedada None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:21.981038 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbbf34a-6bec-4050-9fd9-e8c335969e76 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:22.026707 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd269f8-4009-420c-9c5b-f7933bd60241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:22.027318 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd269f8-4009-420c-9c5b-f7933bd60241 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:22.027599 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd269f8-4009-420c-9c5b-f7933bd60241 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:22.027870 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd269f8-4009-420c-9c5b-f7933bd60241 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:22.028555 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd269f8-4009-420c-9c5b-f7933bd60241 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:22.030487 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd269f8-4009-420c-9c5b-f7933bd60241 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:22.039309 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa1dcf2-7038-451a-9c86-1b063fbedada None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:22.052508 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 20/40] 10.222.0.70 () {60 vars in 1131 bytes} [Thu Oct 27 09:23:21 2022] POST /identity/v3/roles => generated 225 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:23:22.054158 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1e6c6-28a2-47f9-b173-fbeb3cb95e20 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:22.054626 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1e6c6-28a2-47f9-b173-fbeb3cb95e20 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:22.055099 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1e6c6-28a2-47f9-b173-fbeb3cb95e20 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:22.056910 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 21/41] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:23:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:23:22.063450 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb4795-814e-474b-ba71-4e36b29f788a None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:22.064042 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb4795-814e-474b-ba71-4e36b29f788a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:22.064551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb4795-814e-474b-ba71-4e36b29f788a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:22.086355 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-0fbb4795-814e-474b-ba71-4e36b29f788a None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:23:22.087344 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcda6fb-910a-4331-b2be-88524728685f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:22.088126 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcda6fb-910a-4331-b2be-88524728685f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:22.088605 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcda6fb-910a-4331-b2be-88524728685f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:22.088980 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0fbb4795-814e-474b-ba71-4e36b29f788a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:23:22.089090 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcda6fb-910a-4331-b2be-88524728685f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:22.089953 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcda6fb-910a-4331-b2be-88524728685f None admin] RBAC: Adding query filter params (role.id=0dfd839b7a194aee96f82f4ebc1620a7, scope.domain.id=default, user.id=4e3d5d12b2ba49459e93d0a715783a8a) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:23:22.090490 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcda6fb-910a-4331-b2be-88524728685f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:22.092106 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcda6fb-910a-4331-b2be-88524728685f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:22.106119 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 21/42] 10.222.0.70 () {56 vars in 1322 bytes} [Thu Oct 27 09:23:21 2022] GET /identity/v3/role_assignments?user.id=4e3d5d12b2ba49459e93d0a715783a8a&scope.domain.id=default&role.id=0dfd839b7a194aee96f82f4ebc1620a7 => generated 530 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:22.107712 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcda6fb-910a-4331-b2be-88524728685f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:22.122761 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:22.180006 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:22.251154 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcda6fb-910a-4331-b2be-88524728685f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:22.300963 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 22/43] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:23:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 27 09:23:22.305321 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc71d411-7383-4643-902d-1339f2138d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:22.305793 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71d411-7383-4643-902d-1339f2138d43 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:22.306228 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71d411-7383-4643-902d-1339f2138d43 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:22.306329 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71d411-7383-4643-902d-1339f2138d43 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:22.308005 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc71d411-7383-4643-902d-1339f2138d43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:22.309508 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc71d411-7383-4643-902d-1339f2138d43 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:22.311069 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 22/44] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:23:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:22.317128 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb4795-814e-474b-ba71-4e36b29f788a None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:22.427570 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb4795-814e-474b-ba71-4e36b29f788a None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:22.461409 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586bb3d4-8d2f-4cf5-bff3-a6ad2aaee79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:22.462019 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586bb3d4-8d2f-4cf5-bff3-a6ad2aaee79a None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:22.462303 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586bb3d4-8d2f-4cf5-bff3-a6ad2aaee79a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:22.462581 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586bb3d4-8d2f-4cf5-bff3-a6ad2aaee79a None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:22.463246 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586bb3d4-8d2f-4cf5-bff3-a6ad2aaee79a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:22.464780 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586bb3d4-8d2f-4cf5-bff3-a6ad2aaee79a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:22.492806 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 23/45] 10.222.0.70 () {60 vars in 1141 bytes} [Thu Oct 27 09:23:22 2022] POST /identity/v3/projects => generated 312 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:23:22.570466 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc71d411-7383-4643-902d-1339f2138d43 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:22.674249 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc71d411-7383-4643-902d-1339f2138d43 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:22.687494 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:22.708163 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6df974-20d5-4ce5-ac8d-ff4119151aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:22.708798 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6df974-20d5-4ce5-ac8d-ff4119151aa3 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:22.709077 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6df974-20d5-4ce5-ac8d-ff4119151aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:22.709345 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6df974-20d5-4ce5-ac8d-ff4119151aa3 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:22.710005 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6df974-20d5-4ce5-ac8d-ff4119151aa3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:22.711364 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6df974-20d5-4ce5-ac8d-ff4119151aa3 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:22.829431 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 23/46] 10.222.0.70 () {60 vars in 1131 bytes} [Thu Oct 27 09:23:22 2022] POST /identity/v3/roles => generated 223 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:23:24.670533 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a9a38-4c7e-47f2-96f4-d216b41ca270 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:24.671045 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a9a38-4c7e-47f2-96f4-d216b41ca270 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:24.671142 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a9a38-4c7e-47f2-96f4-d216b41ca270 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:24.672424 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 24/47] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:23:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:23:24.677930 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeca6e6-5337-4cf6-9408-bac48fc428b5 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:24.678296 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeca6e6-5337-4cf6-9408-bac48fc428b5 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:24.678486 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeca6e6-5337-4cf6-9408-bac48fc428b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:24.697255 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-5eeca6e6-5337-4cf6-9408-bac48fc428b5 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:23:24.698810 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5eeca6e6-5337-4cf6-9408-bac48fc428b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:23:24.762299 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 24/48] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:23:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:23:25.031887 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5a9a38-4c7e-47f2-96f4-d216b41ca270 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:25.048375 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:25.133616 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5a9a38-4c7e-47f2-96f4-d216b41ca270 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:25.166447 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9129ac60-c6ac-4c1d-95ff-6fcdcead2213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:25.167668 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129ac60-c6ac-4c1d-95ff-6fcdcead2213 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:25.167948 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129ac60-c6ac-4c1d-95ff-6fcdcead2213 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:25.168221 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129ac60-c6ac-4c1d-95ff-6fcdcead2213 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:25.169377 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9129ac60-c6ac-4c1d-95ff-6fcdcead2213 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:25.170807 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9129ac60-c6ac-4c1d-95ff-6fcdcead2213 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:25.172350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 25/49] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:23:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:25.177749 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeca6e6-5337-4cf6-9408-bac48fc428b5 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:25.273277 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeca6e6-5337-4cf6-9408-bac48fc428b5 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:25.274769 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:25.308860 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8236839-2985-4c57-bebf-ac1a2b2b0001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:25.309469 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8236839-2985-4c57-bebf-ac1a2b2b0001 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:25.309748 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8236839-2985-4c57-bebf-ac1a2b2b0001 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:25.310019 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8236839-2985-4c57-bebf-ac1a2b2b0001 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:25.310672 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8236839-2985-4c57-bebf-ac1a2b2b0001 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:25.312237 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8236839-2985-4c57-bebf-ac1a2b2b0001 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:25.337435 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 25/50] 10.222.0.70 () {60 vars in 1140 bytes} [Thu Oct 27 09:23:25 2022] POST /identity/v3/projects => generated 298 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:23:27.464757 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d3d7a-7a98-41e6-9f4a-ef901f0ed666 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:27.465287 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d3d7a-7a98-41e6-9f4a-ef901f0ed666 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:27.465287 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d3d7a-7a98-41e6-9f4a-ef901f0ed666 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:27.466639 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 26/51] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:23:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:23:27.472165 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68acdbb-ed3e-4719-99fc-7a66df7990e5 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:27.472558 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68acdbb-ed3e-4719-99fc-7a66df7990e5 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:27.472743 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68acdbb-ed3e-4719-99fc-7a66df7990e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:27.593682 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f68acdbb-ed3e-4719-99fc-7a66df7990e5 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:23:27.594888 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f68acdbb-ed3e-4719-99fc-7a66df7990e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:23:27.659051 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 26/52] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:23:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:23:27.927954 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d3d7a-7a98-41e6-9f4a-ef901f0ed666 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:28.009945 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:28.028979 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d3d7a-7a98-41e6-9f4a-ef901f0ed666 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:28.062959 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc01ffd4-422b-4de7-80b0-a904b59b504e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:28.063860 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01ffd4-422b-4de7-80b0-a904b59b504e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:28.064142 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01ffd4-422b-4de7-80b0-a904b59b504e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:28.064416 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01ffd4-422b-4de7-80b0-a904b59b504e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:28.065569 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc01ffd4-422b-4de7-80b0-a904b59b504e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:28.067022 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc01ffd4-422b-4de7-80b0-a904b59b504e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:28.068592 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 27/53] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:23:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:28.074901 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68acdbb-ed3e-4719-99fc-7a66df7990e5 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:28.154246 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:28.170559 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68acdbb-ed3e-4719-99fc-7a66df7990e5 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:28.203328 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f8405e-eb1f-4bd4-a5cd-3c717be58193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:28.203871 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8405e-eb1f-4bd4-a5cd-3c717be58193 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:28.204149 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8405e-eb1f-4bd4-a5cd-3c717be58193 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:28.204419 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8405e-eb1f-4bd4-a5cd-3c717be58193 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:28.205084 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f8405e-eb1f-4bd4-a5cd-3c717be58193 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:28.206502 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f8405e-eb1f-4bd4-a5cd-3c717be58193 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:28.232821 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 27/54] 10.222.0.70 () {60 vars in 1132 bytes} [Thu Oct 27 09:23:28 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:23:31.522197 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91cf524-e16a-4d9c-9955-8ad47f88968c None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:31.523193 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91cf524-e16a-4d9c-9955-8ad47f88968c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:31.523677 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91cf524-e16a-4d9c-9955-8ad47f88968c None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:31.525256 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 28/55] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:23:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:23:31.531291 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f05dc-5e54-491a-a441-1c6bfa7186ce None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:31.531687 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f05dc-5e54-491a-a441-1c6bfa7186ce None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:31.531835 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f05dc-5e54-491a-a441-1c6bfa7186ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:31.556875 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-038f05dc-5e54-491a-a441-1c6bfa7186ce None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:23:31.558339 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-038f05dc-5e54-491a-a441-1c6bfa7186ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:23:31.601142 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e78e350-3ad9-4a38-88b1-55670d81b12c None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:31.601904 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e78e350-3ad9-4a38-88b1-55670d81b12c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:31.603026 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e78e350-3ad9-4a38-88b1-55670d81b12c None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:31.605218 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 29/56] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:23:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:23:31.612995 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c5568-d667-4255-8508-b67268c8d7de None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:31.614521 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c5568-d667-4255-8508-b67268c8d7de None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:31.615025 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c5568-d667-4255-8508-b67268c8d7de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:31.640621 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-792c5568-d667-4255-8508-b67268c8d7de None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:23:31.642164 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-792c5568-d667-4255-8508-b67268c8d7de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:23:31.671218 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 28/57] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:23:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:23:31.736455 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 30/58] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:23:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:23:31.758652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc464c-d8c0-4863-8dd9-66696e7a4758 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:31.758969 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc464c-d8c0-4863-8dd9-66696e7a4758 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:31.759212 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc464c-d8c0-4863-8dd9-66696e7a4758 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:31.761254 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 29/59] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:23:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:23:31.772959 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f785b833-8ad4-4fd6-8931-1779ad0f107c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:31.773245 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f785b833-8ad4-4fd6-8931-1779ad0f107c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:31.773520 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f785b833-8ad4-4fd6-8931-1779ad0f107c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:31.795909 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-f785b833-8ad4-4fd6-8931-1779ad0f107c None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:23:31.797135 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f785b833-8ad4-4fd6-8931-1779ad0f107c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:23:31.895883 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 31/60] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:23:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:23:31.970094 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc464c-d8c0-4863-8dd9-66696e7a4758 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:32.063619 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:32.093693 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94da1b2-ecb5-42a0-802d-74afe5ebed95 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:32.093979 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94da1b2-ecb5-42a0-802d-74afe5ebed95 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:32.094428 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94da1b2-ecb5-42a0-802d-74afe5ebed95 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:32.096272 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 32/61] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:23:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:23:32.104773 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8db1a7-af09-45f3-b451-951adc31c9df None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:32.105056 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8db1a7-af09-45f3-b451-951adc31c9df None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:32.105490 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8db1a7-af09-45f3-b451-951adc31c9df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:32.116548 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc464c-d8c0-4863-8dd9-66696e7a4758 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:32.129295 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-4d8db1a7-af09-45f3-b451-951adc31c9df None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:23:32.130864 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4d8db1a7-af09-45f3-b451-951adc31c9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:23:32.152981 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:32.157916 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3301481-6117-425e-9017-3aeb27b50c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:32.158736 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3301481-6117-425e-9017-3aeb27b50c3b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:32.159214 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3301481-6117-425e-9017-3aeb27b50c3b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:32.159630 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3301481-6117-425e-9017-3aeb27b50c3b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:32.172406 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3301481-6117-425e-9017-3aeb27b50c3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:23:32.177758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-a3301481-6117-425e-9017-3aeb27b50c3b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 27 09:23:32.177758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:23:32.177758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:23:32.177758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:23:32.177758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:23:32.177758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:23:32.177758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:23:32.177758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:23:32.177758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:23:32.177758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:23:32.177758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:23:32.177758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:23:32.177758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:23:32.177758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:23:32.177758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:23:32.177758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:23:32.178784 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:23:32.178784 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:23:32.178784 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:23:32.178784 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:23:32.178784 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:23:32.178784 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:23:32.178784 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:23:32.178784 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:23:32.178784 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:23:32.178784 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:23:32.178784 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:23:32.178784 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:23:32.178784 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:23:32.178784 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:23:32.178784 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:23:32.178784 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:23:32.179719 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:23:32.179719 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:23:32.179719 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:23:32.179719 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:23:32.179719 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:23:32.179719 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:23:32.179719 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:23:32.179719 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:23:32.179719 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:23:32.179719 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:23:32.179719 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 27 09:23:32.179719 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:23:32.181778 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 30/62] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Oct 27 09:23:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:23:32.183745 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3301481-6117-425e-9017-3aeb27b50c3b None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:32.212223 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 33/63] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:23:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 27 09:23:32.213523 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8db1a7-af09-45f3-b451-951adc31c9df None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:32.289564 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3301481-6117-425e-9017-3aeb27b50c3b None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:32.320548 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:32.327541 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec8862e-66da-4594-833a-98866b06ed0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:32.328173 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8862e-66da-4594-833a-98866b06ed0f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:32.328453 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8862e-66da-4594-833a-98866b06ed0f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:32.328754 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8862e-66da-4594-833a-98866b06ed0f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:32.328965 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8db1a7-af09-45f3-b451-951adc31c9df None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:32.341517 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec8862e-66da-4594-833a-98866b06ed0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:23:32.346592 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-bec8862e-66da-4594-833a-98866b06ed0f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 09:23:32.346592 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:23:32.346592 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:23:32.346592 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:23:32.346592 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:23:32.346592 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:23:32.346592 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:23:32.346592 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:23:32.346592 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:23:32.346592 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:23:32.346592 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:23:32.346592 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:23:32.346592 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:23:32.346592 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:23:32.346592 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:23:32.346592 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:23:32.347452 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:23:32.347452 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:23:32.347452 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:23:32.347452 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:23:32.347452 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:23:32.347452 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:23:32.347452 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:23:32.347452 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:23:32.347452 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:23:32.347452 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:23:32.347452 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:23:32.347452 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:23:32.347452 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:23:32.347452 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:23:32.347452 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:23:32.347452 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:23:32.348231 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:23:32.348231 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:23:32.348231 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:23:32.348231 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:23:32.348231 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:23:32.348231 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:23:32.348231 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:23:32.348231 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:23:32.348231 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:23:32.348231 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:23:32.348231 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 09:23:32.348231 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:23:32.348231 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 31/64] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Oct 27 09:23:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:23:32.349457 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3f947-0f6e-42de-8c0f-240bef2d77ca None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:32.349457 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3f947-0f6e-42de-8c0f-240bef2d77ca None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:32.349457 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3f947-0f6e-42de-8c0f-240bef2d77ca None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:32.355675 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 32/65] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:23:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:23:32.355675 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b3f947-0f6e-42de-8c0f-240bef2d77ca None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:32.374623 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb78fa-1362-4b22-9c6c-d1865d1d9ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:32.375295 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb78fa-1362-4b22-9c6c-d1865d1d9ca4 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:32.375578 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb78fa-1362-4b22-9c6c-d1865d1d9ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:32.375852 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb78fa-1362-4b22-9c6c-d1865d1d9ca4 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:32.376685 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb78fa-1362-4b22-9c6c-d1865d1d9ca4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:23:32.377107 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb78fa-1362-4b22-9c6c-d1865d1d9ca4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:32.378557 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb78fa-1362-4b22-9c6c-d1865d1d9ca4 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:32.387850 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 34/66] 10.222.0.70 () {56 vars in 1101 bytes} [Thu Oct 27 09:23:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:32.388491 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb78fa-1362-4b22-9c6c-d1865d1d9ca4 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:32.394933 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:32.460197 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b3f947-0f6e-42de-8c0f-240bef2d77ca None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:32.499410 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb78fa-1362-4b22-9c6c-d1865d1d9ca4 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:32.499581 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a325a13a-6e38-4638-b6b3-a45ce55d648d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:32.500204 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325a13a-6e38-4638-b6b3-a45ce55d648d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:32.500485 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325a13a-6e38-4638-b6b3-a45ce55d648d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:32.500786 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325a13a-6e38-4638-b6b3-a45ce55d648d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:32.510779 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a325a13a-6e38-4638-b6b3-a45ce55d648d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:23:32.515531 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-a325a13a-6e38-4638-b6b3-a45ce55d648d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 27 09:23:32.515531 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:23:32.515531 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:23:32.515531 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:23:32.515531 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:23:32.515531 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:23:32.515531 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:23:32.515531 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:23:32.515531 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:23:32.515531 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:23:32.515531 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:23:32.515531 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:23:32.515531 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:23:32.515531 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:23:32.515531 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:23:32.515531 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:23:32.516452 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:23:32.516452 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:23:32.516452 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:23:32.516452 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:23:32.516452 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:23:32.516452 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:23:32.516452 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:23:32.516452 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:23:32.516452 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:23:32.516452 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:23:32.516452 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:23:32.516452 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:23:32.516452 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:23:32.516452 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:23:32.516452 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:23:32.516452 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:23:32.517262 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:23:32.517262 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:23:32.517262 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:23:32.517262 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:23:32.517262 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:23:32.517262 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:23:32.517262 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:23:32.517262 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:23:32.517262 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:23:32.517262 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:23:32.517262 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 27 09:23:32.517262 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:23:32.517262 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 33/67] 10.222.0.70 () {56 vars in 1114 bytes} [Thu Oct 27 09:23:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:23:32.517879 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320773c-2458-4608-b52d-7d008e70442b None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:32.518413 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320773c-2458-4608-b52d-7d008e70442b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:32.518413 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320773c-2458-4608-b52d-7d008e70442b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:32.535067 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac69fb8-bc1a-437a-a6f2-406f912b479d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:32.535680 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac69fb8-bc1a-437a-a6f2-406f912b479d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:32.535813 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-e320773c-2458-4608-b52d-7d008e70442b None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:23:32.536163 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac69fb8-bc1a-437a-a6f2-406f912b479d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:32.536252 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac69fb8-bc1a-437a-a6f2-406f912b479d None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:32.536918 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac69fb8-bc1a-437a-a6f2-406f912b479d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:23:32.537069 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e320773c-2458-4608-b52d-7d008e70442b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:23:32.537224 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac69fb8-bc1a-437a-a6f2-406f912b479d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:32.538610 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac69fb8-bc1a-437a-a6f2-406f912b479d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:32.547338 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 35/68] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Oct 27 09:23:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:32.548125 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac69fb8-bc1a-437a-a6f2-406f912b479d None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:32.585046 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:32.615205 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 34/69] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:23:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:23:32.615205 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e320773c-2458-4608-b52d-7d008e70442b None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:32.621627 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:32.667855 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac69fb8-bc1a-437a-a6f2-406f912b479d None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:32.703229 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c79e79-c753-48fd-8b97-61d3e1799206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:32.703961 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c79e79-c753-48fd-8b97-61d3e1799206 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:32.704238 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c79e79-c753-48fd-8b97-61d3e1799206 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:32.704505 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c79e79-c753-48fd-8b97-61d3e1799206 None admin] PATH_INFO: `/v3/users/483ee2e1417c419482b7963c8e6dd62b` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:32.706854 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e320773c-2458-4608-b52d-7d008e70442b None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:32.719062 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c79e79-c753-48fd-8b97-61d3e1799206 None admin] RBAC: Authorizing `identity:get_user(user_id=483ee2e1417c419482b7963c8e6dd62b)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:32.720953 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c79e79-c753-48fd-8b97-61d3e1799206 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:32.722752 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 36/70] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Oct 27 09:23:32 2022] GET /identity/v3/users/483ee2e1417c419482b7963c8e6dd62b => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:32.723318 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c79e79-c753-48fd-8b97-61d3e1799206 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:32.746983 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9496766f-5919-4d74-bffb-56f625738020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:32.747617 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9496766f-5919-4d74-bffb-56f625738020 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:32.747901 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9496766f-5919-4d74-bffb-56f625738020 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:32.748183 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9496766f-5919-4d74-bffb-56f625738020 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:32.758320 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9496766f-5919-4d74-bffb-56f625738020 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:23:32.764070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-9496766f-5919-4d74-bffb-56f625738020 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 27 09:23:32.764070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:23:32.764070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:23:32.764070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:23:32.764070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:23:32.764070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:23:32.764070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:23:32.764070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:23:32.764070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:23:32.764070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:23:32.764070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:23:32.764070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:23:32.764070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:23:32.764070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:23:32.764070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:23:32.764070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:23:32.764951 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:23:32.764951 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:23:32.764951 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:23:32.764951 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:23:32.764951 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:23:32.764951 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:23:32.764951 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:23:32.764951 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:23:32.764951 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:23:32.764951 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:23:32.764951 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:23:32.764951 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:23:32.764951 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:23:32.764951 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:23:32.764951 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:23:32.764951 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:23:32.765706 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:23:32.765706 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:23:32.765706 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:23:32.765706 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:23:32.765706 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:23:32.765706 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:23:32.765706 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:23:32.765706 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:23:32.765706 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:23:32.765706 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:23:32.765706 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 27 09:23:32.765706 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:23:32.765706 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 35/71] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Oct 27 09:23:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:23:32.766323 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9496766f-5919-4d74-bffb-56f625738020 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:32.810728 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:32.837332 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c79e79-c753-48fd-8b97-61d3e1799206 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:32.862226 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:32.870954 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9496766f-5919-4d74-bffb-56f625738020 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:32.874965 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68444215-46e9-4bb7-8a27-be80aa87da35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:32.875682 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68444215-46e9-4bb7-8a27-be80aa87da35 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:32.876083 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68444215-46e9-4bb7-8a27-be80aa87da35 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:32.876454 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68444215-46e9-4bb7-8a27-be80aa87da35 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:32.877234 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68444215-46e9-4bb7-8a27-be80aa87da35 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:23:32.877635 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68444215-46e9-4bb7-8a27-be80aa87da35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:32.879111 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68444215-46e9-4bb7-8a27-be80aa87da35 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:32.887886 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 37/72] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Oct 27 09:23:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:32.888604 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68444215-46e9-4bb7-8a27-be80aa87da35 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:32.910629 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c008909e-de18-4be2-9bd9-e42df6783c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:32.911262 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008909e-de18-4be2-9bd9-e42df6783c75 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:32.911545 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008909e-de18-4be2-9bd9-e42df6783c75 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:32.911823 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008909e-de18-4be2-9bd9-e42df6783c75 None admin] PATH_INFO: `/v3/users/4e3d5d12b2ba49459e93d0a715783a8a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:32.912988 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c008909e-de18-4be2-9bd9-e42df6783c75 None admin] RBAC: Authorizing `identity:get_user(user_id=4e3d5d12b2ba49459e93d0a715783a8a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:32.914455 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c008909e-de18-4be2-9bd9-e42df6783c75 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:32.915999 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 36/73] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Oct 27 09:23:32 2022] GET /identity/v3/users/4e3d5d12b2ba49459e93d0a715783a8a => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:32.920435 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c008909e-de18-4be2-9bd9-e42df6783c75 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:32.990250 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68444215-46e9-4bb7-8a27-be80aa87da35 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:33.017055 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c008909e-de18-4be2-9bd9-e42df6783c75 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:33.023141 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493fa973-e01c-4ad7-bf27-a8ae84a9e7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:33.023769 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493fa973-e01c-4ad7-bf27-a8ae84a9e7f3 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:33.024042 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493fa973-e01c-4ad7-bf27-a8ae84a9e7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:33.024307 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493fa973-e01c-4ad7-bf27-a8ae84a9e7f3 None admin] PATH_INFO: `/v3/projects/78bd002b14ac4653ae0975911fcabf81` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:33.026154 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493fa973-e01c-4ad7-bf27-a8ae84a9e7f3 None admin] RBAC: Authorizing `identity:get_project(project_id=78bd002b14ac4653ae0975911fcabf81)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:33.026655 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:33.027662 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493fa973-e01c-4ad7-bf27-a8ae84a9e7f3 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:33.029421 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 38/74] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Oct 27 09:23:32 2022] GET /identity/v3/projects/78bd002b14ac4653ae0975911fcabf81 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:33.030108 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493fa973-e01c-4ad7-bf27-a8ae84a9e7f3 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:33.052362 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c52676-1919-4c05-9388-04bbf6c1cfb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:33.053001 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c52676-1919-4c05-9388-04bbf6c1cfb1 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:33.053283 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c52676-1919-4c05-9388-04bbf6c1cfb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:33.053557 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c52676-1919-4c05-9388-04bbf6c1cfb1 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:33.054218 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c52676-1919-4c05-9388-04bbf6c1cfb1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:23:33.054524 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c52676-1919-4c05-9388-04bbf6c1cfb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:33.055910 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c52676-1919-4c05-9388-04bbf6c1cfb1 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:33.063899 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 37/75] 10.222.0.70 () {56 vars in 1101 bytes} [Thu Oct 27 09:23:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:33.064793 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c52676-1919-4c05-9388-04bbf6c1cfb1 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:33.085699 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:33.133041 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493fa973-e01c-4ad7-bf27-a8ae84a9e7f3 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:33.164979 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c52676-1919-4c05-9388-04bbf6c1cfb1 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:33.166333 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88085770-9dba-4fae-9130-26351c214053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:33.166964 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88085770-9dba-4fae-9130-26351c214053 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:33.167253 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88085770-9dba-4fae-9130-26351c214053 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:33.167531 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88085770-9dba-4fae-9130-26351c214053 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:33.168692 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88085770-9dba-4fae-9130-26351c214053 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:33.170152 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88085770-9dba-4fae-9130-26351c214053 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:33.171709 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 39/76] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:23:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:33.172471 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88085770-9dba-4fae-9130-26351c214053 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:33.199076 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ae1751-2989-453e-9ada-187f83e74c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:33.199698 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae1751-2989-453e-9ada-187f83e74c79 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:33.199981 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae1751-2989-453e-9ada-187f83e74c79 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:33.200252 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae1751-2989-453e-9ada-187f83e74c79 None admin] PATH_INFO: `/v3/users/483ee2e1417c419482b7963c8e6dd62b` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:33.201412 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ae1751-2989-453e-9ada-187f83e74c79 None admin] RBAC: Authorizing `identity:get_user(user_id=483ee2e1417c419482b7963c8e6dd62b)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:33.202851 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ae1751-2989-453e-9ada-187f83e74c79 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:33.204403 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 38/77] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Oct 27 09:23:33 2022] GET /identity/v3/users/483ee2e1417c419482b7963c8e6dd62b => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:33.205165 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ae1751-2989-453e-9ada-187f83e74c79 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:33.242524 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:33.278481 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88085770-9dba-4fae-9130-26351c214053 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:33.299735 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:33.307770 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ae1751-2989-453e-9ada-187f83e74c79 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:33.314546 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca352be5-ce44-4144-bee4-5072ff573c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:33.315171 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca352be5-ce44-4144-bee4-5072ff573c67 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:33.315447 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca352be5-ce44-4144-bee4-5072ff573c67 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:33.315713 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca352be5-ce44-4144-bee4-5072ff573c67 None admin] PATH_INFO: `/v3/projects/78bd002b14ac4653ae0975911fcabf81` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:33.316860 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca352be5-ce44-4144-bee4-5072ff573c67 None admin] RBAC: Authorizing `identity:get_project(project_id=78bd002b14ac4653ae0975911fcabf81)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:33.318312 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca352be5-ce44-4144-bee4-5072ff573c67 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:33.320115 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 40/78] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Oct 27 09:23:33 2022] GET /identity/v3/projects/78bd002b14ac4653ae0975911fcabf81 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:33.320699 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca352be5-ce44-4144-bee4-5072ff573c67 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:33.342262 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b8eff-10cc-4b8a-ae57-1dc6f8d77dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:33.342925 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b8eff-10cc-4b8a-ae57-1dc6f8d77dee None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:33.343207 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b8eff-10cc-4b8a-ae57-1dc6f8d77dee None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:33.343486 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b8eff-10cc-4b8a-ae57-1dc6f8d77dee None admin] PATH_INFO: `/v3/projects/78bd002b14ac4653ae0975911fcabf81/users/483ee2e1417c419482b7963c8e6dd62b/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:33.345445 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b8eff-10cc-4b8a-ae57-1dc6f8d77dee None admin] RBAC: Authorizing `identity:create_grant(project_id=78bd002b14ac4653ae0975911fcabf81, user_id=483ee2e1417c419482b7963c8e6dd62b, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:33.347231 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b8eff-10cc-4b8a-ae57-1dc6f8d77dee None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:33.360751 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 39/79] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Oct 27 09:23:33 2022] PUT /identity/v3/projects/78bd002b14ac4653ae0975911fcabf81/users/483ee2e1417c419482b7963c8e6dd62b/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:23:33.361528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b8eff-10cc-4b8a-ae57-1dc6f8d77dee None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:33.428701 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca352be5-ce44-4144-bee4-5072ff573c67 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:33.466915 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9824a9c-b953-4279-953f-858c55c3ea28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:33.467602 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9824a9c-b953-4279-953f-858c55c3ea28 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:33.467884 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9824a9c-b953-4279-953f-858c55c3ea28 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:33.468161 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9824a9c-b953-4279-953f-858c55c3ea28 None admin] PATH_INFO: `/v3/users/483ee2e1417c419482b7963c8e6dd62b` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:33.469421 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9824a9c-b953-4279-953f-858c55c3ea28 None admin] RBAC: Authorizing `identity:get_user(user_id=483ee2e1417c419482b7963c8e6dd62b)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:33.471217 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9824a9c-b953-4279-953f-858c55c3ea28 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:33.472892 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 41/80] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Oct 27 09:23:33 2022] GET /identity/v3/users/483ee2e1417c419482b7963c8e6dd62b => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:33.473675 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9824a9c-b953-4279-953f-858c55c3ea28 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:33.496091 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:33.501206 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b8eff-10cc-4b8a-ae57-1dc6f8d77dee None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:33.542308 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:33.544422 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4758d4-072d-49de-92a6-84526a9c2cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:33.545077 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4758d4-072d-49de-92a6-84526a9c2cc7 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:33.545376 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4758d4-072d-49de-92a6-84526a9c2cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:33.545653 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4758d4-072d-49de-92a6-84526a9c2cc7 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:33.546300 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4758d4-072d-49de-92a6-84526a9c2cc7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:33.547786 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4758d4-072d-49de-92a6-84526a9c2cc7 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:33.573198 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 40/81] 10.222.0.70 () {60 vars in 1132 bytes} [Thu Oct 27 09:23:33 2022] POST /identity/v3/users => generated 289 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:23:33.574006 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4758d4-072d-49de-92a6-84526a9c2cc7 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:33.606099 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9824a9c-b953-4279-953f-858c55c3ea28 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:33.646395 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888af1ce-50c2-4ea2-9ec9-40b4f41d1708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:33.647044 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888af1ce-50c2-4ea2-9ec9-40b4f41d1708 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:33.647331 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888af1ce-50c2-4ea2-9ec9-40b4f41d1708 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:33.647610 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888af1ce-50c2-4ea2-9ec9-40b4f41d1708 None admin] PATH_INFO: `/v3/projects/78bd002b14ac4653ae0975911fcabf81` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:33.649039 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888af1ce-50c2-4ea2-9ec9-40b4f41d1708 None admin] RBAC: Authorizing `identity:get_project(project_id=78bd002b14ac4653ae0975911fcabf81)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:33.650575 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888af1ce-50c2-4ea2-9ec9-40b4f41d1708 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:33.652791 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 42/82] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Oct 27 09:23:33 2022] GET /identity/v3/projects/78bd002b14ac4653ae0975911fcabf81 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:33.653467 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888af1ce-50c2-4ea2-9ec9-40b4f41d1708 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:33.724936 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4758d4-072d-49de-92a6-84526a9c2cc7 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:33.771426 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888af1ce-50c2-4ea2-9ec9-40b4f41d1708 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:33.774274 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b24ce-f188-404c-bea5-b883462018a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:33.776656 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b24ce-f188-404c-bea5-b883462018a5 None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:33.776656 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b24ce-f188-404c-bea5-b883462018a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:33.776656 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b24ce-f188-404c-bea5-b883462018a5 None admin] PATH_INFO: `/v3/projects/78bd002b14ac4653ae0975911fcabf81/users/4e3d5d12b2ba49459e93d0a715783a8a/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:33.779161 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99b24ce-f188-404c-bea5-b883462018a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=78bd002b14ac4653ae0975911fcabf81, user_id=4e3d5d12b2ba49459e93d0a715783a8a, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:33.780133 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99b24ce-f188-404c-bea5-b883462018a5 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:33.797010 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 41/83] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Oct 27 09:23:33 2022] PUT /identity/v3/projects/78bd002b14ac4653ae0975911fcabf81/users/4e3d5d12b2ba49459e93d0a715783a8a/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:23:33.798259 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b24ce-f188-404c-bea5-b883462018a5 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:33.802203 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:33.819140 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:33.821044 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02b5787-708d-4966-a20b-fa8bf0b2b676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:33.821785 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02b5787-708d-4966-a20b-fa8bf0b2b676 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:33.822072 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02b5787-708d-4966-a20b-fa8bf0b2b676 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:33.823559 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02b5787-708d-4966-a20b-fa8bf0b2b676 None admin] PATH_INFO: `/v3/projects/ca239ed7b56748d19640893698c1891c` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:33.824891 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02b5787-708d-4966-a20b-fa8bf0b2b676 None admin] RBAC: Authorizing `identity:get_project(project_id=ca239ed7b56748d19640893698c1891c)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:33.828156 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02b5787-708d-4966-a20b-fa8bf0b2b676 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:33.832771 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 43/84] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Oct 27 09:23:33 2022] GET /identity/v3/projects/ca239ed7b56748d19640893698c1891c => generated 312 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:33.840572 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02b5787-708d-4966-a20b-fa8bf0b2b676 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:33.938936 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b24ce-f188-404c-bea5-b883462018a5 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:33.974079 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02b5787-708d-4966-a20b-fa8bf0b2b676 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:33.990947 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f31d2d8-ab9a-4862-811b-9adc716c5f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:33.990947 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f31d2d8-ab9a-4862-811b-9adc716c5f7f None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:33.990947 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f31d2d8-ab9a-4862-811b-9adc716c5f7f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:33.991304 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f31d2d8-ab9a-4862-811b-9adc716c5f7f None admin] PATH_INFO: `/v3/projects/78bd002b14ac4653ae0975911fcabf81/users/483ee2e1417c419482b7963c8e6dd62b/roles/fa269b0e4dc54a24bb729398ce38231c` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:33.996850 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f31d2d8-ab9a-4862-811b-9adc716c5f7f None admin] RBAC: Authorizing `identity:create_grant(project_id=78bd002b14ac4653ae0975911fcabf81, user_id=483ee2e1417c419482b7963c8e6dd62b, role_id=fa269b0e4dc54a24bb729398ce38231c)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:33.997345 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f31d2d8-ab9a-4862-811b-9adc716c5f7f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:34.014375 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 42/85] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Oct 27 09:23:33 2022] PUT /identity/v3/projects/78bd002b14ac4653ae0975911fcabf81/users/483ee2e1417c419482b7963c8e6dd62b/roles/fa269b0e4dc54a24bb729398ce38231c => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:23:34.028425 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aa452b-7692-4770-a070-42e2b6c02bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:34.029216 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aa452b-7692-4770-a070-42e2b6c02bc5 None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:34.029550 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aa452b-7692-4770-a070-42e2b6c02bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:34.029828 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aa452b-7692-4770-a070-42e2b6c02bc5 None admin] PATH_INFO: `/v3/projects/ca239ed7b56748d19640893698c1891c/users/483ee2e1417c419482b7963c8e6dd62b/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:34.031921 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aa452b-7692-4770-a070-42e2b6c02bc5 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca239ed7b56748d19640893698c1891c, user_id=483ee2e1417c419482b7963c8e6dd62b, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:34.033538 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aa452b-7692-4770-a070-42e2b6c02bc5 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:34.053220 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 44/86] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Oct 27 09:23:33 2022] PUT /identity/v3/projects/ca239ed7b56748d19640893698c1891c/users/483ee2e1417c419482b7963c8e6dd62b/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:23:35.929869 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6915ac-5c09-422d-9928-5c149c4fdc87 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:35.930381 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6915ac-5c09-422d-9928-5c149c4fdc87 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:35.930455 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6915ac-5c09-422d-9928-5c149c4fdc87 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:35.931791 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 43/87] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:23:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:23:35.948068 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73741c8-7e04-492d-806d-989c75637b96 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:35.948944 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73741c8-7e04-492d-806d-989c75637b96 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:35.949421 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73741c8-7e04-492d-806d-989c75637b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:35.974894 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-e73741c8-7e04-492d-806d-989c75637b96 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:23:35.976382 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e73741c8-7e04-492d-806d-989c75637b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:23:36.040358 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:36.067418 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 45/88] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:23:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 27 09:23:36.626215 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6915ac-5c09-422d-9928-5c149c4fdc87 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:36.731015 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:36.748793 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec430a-2606-4fa7-8e49-e5240af65dbb None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:36.749089 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec430a-2606-4fa7-8e49-e5240af65dbb None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:36.753032 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec430a-2606-4fa7-8e49-e5240af65dbb None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:36.754687 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 46/89] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:23:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:23:36.759946 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005dce02-c2f7-4c38-b5b5-aebbdd5ac00e None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:36.760228 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005dce02-c2f7-4c38-b5b5-aebbdd5ac00e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:36.760504 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005dce02-c2f7-4c38-b5b5-aebbdd5ac00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:36.786468 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-005dce02-c2f7-4c38-b5b5-aebbdd5ac00e None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:23:36.787628 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-005dce02-c2f7-4c38-b5b5-aebbdd5ac00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:23:36.791533 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6915ac-5c09-422d-9928-5c149c4fdc87 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:36.863687 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b0d379-81e5-4c68-a4f3-d20168ceea87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:36.864328 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0d379-81e5-4c68-a4f3-d20168ceea87 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:36.864644 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0d379-81e5-4c68-a4f3-d20168ceea87 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:36.864924 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0d379-81e5-4c68-a4f3-d20168ceea87 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:36.889515 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b0d379-81e5-4c68-a4f3-d20168ceea87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:23:36.895006 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-32b0d379-81e5-4c68-a4f3-d20168ceea87 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 27 09:23:36.895006 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:23:36.895006 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:23:36.895006 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:23:36.895006 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:23:36.895006 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:23:36.895006 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:23:36.895006 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:23:36.895006 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:23:36.895006 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:23:36.895006 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:23:36.895006 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:23:36.895006 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:23:36.895006 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:23:36.895006 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:23:36.895006 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:23:36.895864 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:23:36.895864 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:23:36.895864 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:23:36.895864 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:23:36.895864 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:23:36.895864 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:23:36.895864 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:23:36.895864 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:23:36.895864 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:23:36.895864 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:23:36.895864 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:23:36.895864 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:23:36.895864 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:23:36.895864 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:23:36.895864 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:23:36.895864 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:23:36.896638 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:23:36.896638 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:23:36.896638 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:23:36.896638 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:23:36.896638 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:23:36.896638 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:23:36.896638 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:23:36.896638 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:23:36.896638 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:23:36.896638 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:23:36.896638 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 27 09:23:36.896638 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:23:36.896638 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 44/90] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Oct 27 09:23:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:23:36.897266 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89c4474-c382-4bb1-bcbf-25717546d36c None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:36.897817 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89c4474-c382-4bb1-bcbf-25717546d36c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:36.897817 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89c4474-c382-4bb1-bcbf-25717546d36c None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:36.899362 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 45/91] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:23:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:23:36.902535 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 47/92] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:23:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 27 09:23:36.909940 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae17b3f9-9d59-4903-8767-a00357481bde None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:36.910225 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae17b3f9-9d59-4903-8767-a00357481bde None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:36.910501 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae17b3f9-9d59-4903-8767-a00357481bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:36.912923 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005dce02-c2f7-4c38-b5b5-aebbdd5ac00e None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:36.952613 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-ae17b3f9-9d59-4903-8767-a00357481bde None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:23:36.954902 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ae17b3f9-9d59-4903-8767-a00357481bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:23:37.133658 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005dce02-c2f7-4c38-b5b5-aebbdd5ac00e None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:37.149276 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:37.154043 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 46/93] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:23:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:23:37.189939 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1827ae40-feb5-4cd7-b3a9-84c50fe60218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:37.190846 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827ae40-feb5-4cd7-b3a9-84c50fe60218 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:37.191274 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827ae40-feb5-4cd7-b3a9-84c50fe60218 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:37.191425 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c47d87-2daf-4ca9-bf9b-f16b3d742bbb None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:37.191722 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c47d87-2daf-4ca9-bf9b-f16b3d742bbb None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:37.191983 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827ae40-feb5-4cd7-b3a9-84c50fe60218 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:37.192088 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c47d87-2daf-4ca9-bf9b-f16b3d742bbb None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:37.192903 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1827ae40-feb5-4cd7-b3a9-84c50fe60218 None admin] RBAC: Adding query filter params (name=member) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:23:37.193361 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 47/94] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:23:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 27 09:23:37.193541 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1827ae40-feb5-4cd7-b3a9-84c50fe60218 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:37.195140 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1827ae40-feb5-4cd7-b3a9-84c50fe60218 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:37.198932 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c657456-194d-4dfd-9f01-586920d10994 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:37.199216 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c657456-194d-4dfd-9f01-586920d10994 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:37.199492 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c657456-194d-4dfd-9f01-586920d10994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:37.205672 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 48/95] 10.222.0.70 () {56 vars in 1101 bytes} [Thu Oct 27 09:23:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:37.211720 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1827ae40-feb5-4cd7-b3a9-84c50fe60218 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:37.233497 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-7c657456-194d-4dfd-9f01-586920d10994 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:23:37.234669 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7c657456-194d-4dfd-9f01-586920d10994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:23:37.350218 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:37.363656 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 48/96] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:23:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 27 09:23:37.382179 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c657456-194d-4dfd-9f01-586920d10994 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:37.431379 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1827ae40-feb5-4cd7-b3a9-84c50fe60218 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:37.484138 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c657456-194d-4dfd-9f01-586920d10994 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:37.522865 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f324f284-e2d5-468b-adef-08a4a315a4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:37.523517 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324f284-e2d5-468b-adef-08a4a315a4ad None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:37.523793 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324f284-e2d5-468b-adef-08a4a315a4ad None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:37.524062 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324f284-e2d5-468b-adef-08a4a315a4ad None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:37.533236 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13f4586-86ba-4011-88be-1a16af49f37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:37.533889 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f4586-86ba-4011-88be-1a16af49f37d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:37.534169 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f4586-86ba-4011-88be-1a16af49f37d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:37.534441 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f4586-86ba-4011-88be-1a16af49f37d None admin] PATH_INFO: `/v3/users/483ee2e1417c419482b7963c8e6dd62b` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:37.535635 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13f4586-86ba-4011-88be-1a16af49f37d None admin] RBAC: Authorizing `identity:get_user(user_id=483ee2e1417c419482b7963c8e6dd62b)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:37.535758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f324f284-e2d5-468b-adef-08a4a315a4ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:23:37.537132 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13f4586-86ba-4011-88be-1a16af49f37d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:37.538724 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 49/97] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Oct 27 09:23:37 2022] GET /identity/v3/users/483ee2e1417c419482b7963c8e6dd62b => generated 275 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:37.540601 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-f324f284-e2d5-468b-adef-08a4a315a4ad None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 27 09:23:37.540601 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:23:37.540601 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:23:37.540601 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:23:37.540601 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:23:37.540601 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:23:37.540601 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:23:37.540601 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:23:37.540601 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:23:37.540601 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:23:37.540601 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:23:37.540601 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:23:37.540601 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:23:37.540601 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:23:37.540601 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:23:37.540601 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:23:37.542007 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:23:37.542007 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:23:37.542007 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:23:37.542007 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:23:37.542007 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:23:37.542007 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:23:37.542007 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:23:37.542007 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:23:37.542007 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:23:37.542007 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:23:37.542007 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:23:37.542007 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:23:37.542007 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:23:37.542007 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:23:37.542007 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:23:37.542007 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:23:37.543443 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:23:37.543443 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:23:37.543443 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:23:37.543443 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:23:37.543443 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:23:37.543443 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:23:37.543443 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:23:37.543443 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:23:37.543443 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:23:37.543443 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:23:37.543443 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 27 09:23:37.543443 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:23:37.543443 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 49/98] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Oct 27 09:23:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 09:23:37.544971 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13f4586-86ba-4011-88be-1a16af49f37d None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:37.548229 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f324f284-e2d5-468b-adef-08a4a315a4ad None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:37.609249 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:37.643519 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:37.656762 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f324f284-e2d5-468b-adef-08a4a315a4ad None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:37.692680 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89506650-aec1-4e9a-b0f1-4e47e1c4a7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:37.693212 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89506650-aec1-4e9a-b0f1-4e47e1c4a7e9 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:37.693490 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89506650-aec1-4e9a-b0f1-4e47e1c4a7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:37.693762 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89506650-aec1-4e9a-b0f1-4e47e1c4a7e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:37.694416 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89506650-aec1-4e9a-b0f1-4e47e1c4a7e9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:23:37.694717 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89506650-aec1-4e9a-b0f1-4e47e1c4a7e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:37.696156 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89506650-aec1-4e9a-b0f1-4e47e1c4a7e9 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:37.705174 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 50/99] 10.222.0.70 () {56 vars in 1101 bytes} [Thu Oct 27 09:23:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:23:37.705971 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89506650-aec1-4e9a-b0f1-4e47e1c4a7e9 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:37.726199 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13f4586-86ba-4011-88be-1a16af49f37d None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:37.761134 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3b6b89-467f-4b4f-ba1c-c2dfd1cb4fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:37.761744 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b6b89-467f-4b4f-ba1c-c2dfd1cb4fd8 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:37.762437 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b6b89-467f-4b4f-ba1c-c2dfd1cb4fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:37.762711 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b6b89-467f-4b4f-ba1c-c2dfd1cb4fd8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:37.824834 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89506650-aec1-4e9a-b0f1-4e47e1c4a7e9 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:37.829910 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3b6b89-467f-4b4f-ba1c-c2dfd1cb4fd8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:37.831372 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3b6b89-467f-4b4f-ba1c-c2dfd1cb4fd8 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:37.840724 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:37.858801 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e363fc52-b502-45b9-a489-67fd1f6fe323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:37.859430 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363fc52-b502-45b9-a489-67fd1f6fe323 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:37.859710 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363fc52-b502-45b9-a489-67fd1f6fe323 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:37.859981 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363fc52-b502-45b9-a489-67fd1f6fe323 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:37.869817 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e363fc52-b502-45b9-a489-67fd1f6fe323 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:23:37.874199 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:37.875046 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-e363fc52-b502-45b9-a489-67fd1f6fe323 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 09:23:37.875046 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:23:37.875046 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:23:37.875046 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:23:37.875046 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:23:37.875046 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:23:37.875046 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:23:37.875046 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:23:37.875046 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:23:37.875046 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:23:37.875046 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:23:37.875046 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:23:37.875046 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:23:37.875046 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:23:37.875046 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:23:37.875046 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:23:37.875883 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:23:37.875883 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:23:37.875883 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:23:37.875883 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:23:37.875883 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:23:37.875883 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:23:37.875883 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:23:37.875883 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:23:37.875883 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:23:37.875883 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:23:37.875883 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:23:37.875883 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:23:37.875883 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:23:37.875883 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:23:37.875883 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:23:37.875883 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:23:37.881546 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:23:37.881546 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:23:37.881546 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:23:37.881546 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:23:37.881546 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:23:37.881546 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:23:37.881546 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:23:37.881546 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:23:37.881546 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:23:37.881546 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:23:37.881546 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 09:23:37.881546 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:23:37.881546 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 51/100] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Oct 27 09:23:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:23:37.881546 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1775f30-d7be-4c32-9e14-3a6e73e61622 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:37.882219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1775f30-d7be-4c32-9e14-3a6e73e61622 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:37.882219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1775f30-d7be-4c32-9e14-3a6e73e61622 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:37.882219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 52/101] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:23:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:23:37.882219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1775f30-d7be-4c32-9e14-3a6e73e61622 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:37.938907 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 50/102] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Oct 27 09:23:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 27 09:23:37.938907 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c79480a-5278-461b-92fe-1eec413f4797 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:37.938907 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c79480a-5278-461b-92fe-1eec413f4797 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:37.938907 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c79480a-5278-461b-92fe-1eec413f4797 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:37.938907 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 51/103] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:23:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:23:37.938907 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c79480a-5278-461b-92fe-1eec413f4797 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:37.987453 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1775f30-d7be-4c32-9e14-3a6e73e61622 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:38.020714 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302968be-209d-490b-92b0-45db90ab1fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:38.021343 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302968be-209d-490b-92b0-45db90ab1fd3 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:38.021626 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302968be-209d-490b-92b0-45db90ab1fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:38.021903 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302968be-209d-490b-92b0-45db90ab1fd3 None admin] PATH_INFO: `/v3/users/22734224a3904550ae5d228ad14143be` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:38.037550 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302968be-209d-490b-92b0-45db90ab1fd3 None admin] RBAC: Authorizing `identity:get_user(user_id=22734224a3904550ae5d228ad14143be)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:38.039000 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302968be-209d-490b-92b0-45db90ab1fd3 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:38.040626 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 53/104] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Oct 27 09:23:37 2022] GET /identity/v3/users/22734224a3904550ae5d228ad14143be => generated 289 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:38.041474 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302968be-209d-490b-92b0-45db90ab1fd3 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:38.045745 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c79480a-5278-461b-92fe-1eec413f4797 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:38.076943 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:38.079414 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52482ed1-900f-44df-a45f-9d1014d8df72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:38.080049 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52482ed1-900f-44df-a45f-9d1014d8df72 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:38.080329 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52482ed1-900f-44df-a45f-9d1014d8df72 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:38.080632 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52482ed1-900f-44df-a45f-9d1014d8df72 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:38.091457 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52482ed1-900f-44df-a45f-9d1014d8df72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:23:38.096531 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-52482ed1-900f-44df-a45f-9d1014d8df72 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 27 09:23:38.096531 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:23:38.096531 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:23:38.096531 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:23:38.096531 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:23:38.096531 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:23:38.096531 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:23:38.096531 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:23:38.096531 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:23:38.096531 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:23:38.096531 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:23:38.096531 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:23:38.096531 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:23:38.096531 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:23:38.096531 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:23:38.096531 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:23:38.097341 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:23:38.097341 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:23:38.097341 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:23:38.097341 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:23:38.097341 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:23:38.097341 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:23:38.097341 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:23:38.097341 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:23:38.097341 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:23:38.097341 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:23:38.097341 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:23:38.097341 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:23:38.097341 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:23:38.097341 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:23:38.097341 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:23:38.097341 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:23:38.098105 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:23:38.098105 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:23:38.098105 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:23:38.098105 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:23:38.098105 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:23:38.098105 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:23:38.098105 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:23:38.098105 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:23:38.098105 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:23:38.098105 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:23:38.098105 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 27 09:23:38.098105 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:23:38.098105 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 52/105] 10.222.0.70 () {56 vars in 1114 bytes} [Thu Oct 27 09:23:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:23:38.099821 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de9375-c077-4d67-b88b-6b0af5fded12 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:38.100106 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de9375-c077-4d67-b88b-6b0af5fded12 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:38.100381 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de9375-c077-4d67-b88b-6b0af5fded12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:38.107568 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:38.119534 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-e9de9375-c077-4d67-b88b-6b0af5fded12 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:23:38.120733 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e9de9375-c077-4d67-b88b-6b0af5fded12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:23:38.149011 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302968be-209d-490b-92b0-45db90ab1fd3 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:38.183205 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82818ad-d79d-4c65-9d50-39de412733cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:38.183807 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82818ad-d79d-4c65-9d50-39de412733cf None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:38.184089 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82818ad-d79d-4c65-9d50-39de412733cf None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:38.184361 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82818ad-d79d-4c65-9d50-39de412733cf None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:38.185053 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82818ad-d79d-4c65-9d50-39de412733cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:23:38.185356 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82818ad-d79d-4c65-9d50-39de412733cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:38.186061 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 53/106] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:23:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:23:38.186715 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82818ad-d79d-4c65-9d50-39de412733cf None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:38.186879 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9de9375-c077-4d67-b88b-6b0af5fded12 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:38.205336 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 54/107] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Oct 27 09:23:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:38.206562 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede9b69-f7ba-4a81-88ee-8e338a3a0cf9 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:38.206852 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede9b69-f7ba-4a81-88ee-8e338a3a0cf9 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:38.207130 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede9b69-f7ba-4a81-88ee-8e338a3a0cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:38.225301 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-2ede9b69-f7ba-4a81-88ee-8e338a3a0cf9 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:23:38.226506 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2ede9b69-f7ba-4a81-88ee-8e338a3a0cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:23:38.301497 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 55/108] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:23:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:23:38.303302 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9de9375-c077-4d67-b88b-6b0af5fded12 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:38.313509 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede9b69-f7ba-4a81-88ee-8e338a3a0cf9 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:38.334703 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:38.341997 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a98815-cfd5-4956-a30c-1d04ed0aafa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:38.342628 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a98815-cfd5-4956-a30c-1d04ed0aafa8 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:38.342907 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a98815-cfd5-4956-a30c-1d04ed0aafa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:38.343181 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a98815-cfd5-4956-a30c-1d04ed0aafa8 None admin] PATH_INFO: `/v3/projects/78bd002b14ac4653ae0975911fcabf81` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:38.344405 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a98815-cfd5-4956-a30c-1d04ed0aafa8 None admin] RBAC: Authorizing `identity:get_project(project_id=78bd002b14ac4653ae0975911fcabf81)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:38.346267 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a98815-cfd5-4956-a30c-1d04ed0aafa8 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:38.348133 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 54/109] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Oct 27 09:23:38 2022] GET /identity/v3/projects/78bd002b14ac4653ae0975911fcabf81 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:38.353765 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a98815-cfd5-4956-a30c-1d04ed0aafa8 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:38.355308 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:38.451842 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede9b69-f7ba-4a81-88ee-8e338a3a0cf9 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:38.463587 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a98815-cfd5-4956-a30c-1d04ed0aafa8 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:38.498131 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d698fd-8b0b-4950-887a-190b2403f8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:38.498511 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bcd148-b455-41d4-84e5-f1f4e3ed15d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:38.498739 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d698fd-8b0b-4950-887a-190b2403f8ad None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:38.499016 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d698fd-8b0b-4950-887a-190b2403f8ad None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:38.499171 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bcd148-b455-41d4-84e5-f1f4e3ed15d1 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:38.499280 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d698fd-8b0b-4950-887a-190b2403f8ad None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:38.499472 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bcd148-b455-41d4-84e5-f1f4e3ed15d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:38.499750 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bcd148-b455-41d4-84e5-f1f4e3ed15d1 None admin] PATH_INFO: `/v3/projects/78bd002b14ac4653ae0975911fcabf81` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:38.499926 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d698fd-8b0b-4950-887a-190b2403f8ad None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:23:38.500270 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d698fd-8b0b-4950-887a-190b2403f8ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:38.500945 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bcd148-b455-41d4-84e5-f1f4e3ed15d1 None admin] RBAC: Authorizing `identity:get_project(project_id=78bd002b14ac4653ae0975911fcabf81)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:38.501633 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d698fd-8b0b-4950-887a-190b2403f8ad None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:38.502438 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bcd148-b455-41d4-84e5-f1f4e3ed15d1 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:38.504059 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 56/110] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Oct 27 09:23:38 2022] GET /identity/v3/projects/78bd002b14ac4653ae0975911fcabf81 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:38.504831 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bcd148-b455-41d4-84e5-f1f4e3ed15d1 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:38.520101 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 55/111] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Oct 27 09:23:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:38.520101 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d698fd-8b0b-4950-887a-190b2403f8ad None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:38.585083 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:38.607502 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:38.614832 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bcd148-b455-41d4-84e5-f1f4e3ed15d1 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:38.623894 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d698fd-8b0b-4950-887a-190b2403f8ad None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:38.650544 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8041107-2d97-4f83-8fd7-00b251cf08c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:38.651164 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8041107-2d97-4f83-8fd7-00b251cf08c7 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:38.651442 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8041107-2d97-4f83-8fd7-00b251cf08c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:38.651718 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8041107-2d97-4f83-8fd7-00b251cf08c7 None admin] PATH_INFO: `/v3/users/4e3d5d12b2ba49459e93d0a715783a8a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:38.652908 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8041107-2d97-4f83-8fd7-00b251cf08c7 None admin] RBAC: Authorizing `identity:get_user(user_id=4e3d5d12b2ba49459e93d0a715783a8a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:38.654343 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8041107-2d97-4f83-8fd7-00b251cf08c7 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:38.655897 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 57/112] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Oct 27 09:23:38 2022] GET /identity/v3/users/4e3d5d12b2ba49459e93d0a715783a8a => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:38.656678 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8041107-2d97-4f83-8fd7-00b251cf08c7 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:38.657215 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ffc360-7bc1-4a54-8efc-23ebea29ca11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:38.657827 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ffc360-7bc1-4a54-8efc-23ebea29ca11 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:38.658111 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ffc360-7bc1-4a54-8efc-23ebea29ca11 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:38.658388 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ffc360-7bc1-4a54-8efc-23ebea29ca11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:38.659112 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ffc360-7bc1-4a54-8efc-23ebea29ca11 None admin] RBAC: Adding query filter params (role.id=eea7cd23e01b4506a9eb6ab86f9adcd2, scope.project.id=78bd002b14ac4653ae0975911fcabf81, user.id=483ee2e1417c419482b7963c8e6dd62b) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:23:38.659420 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ffc360-7bc1-4a54-8efc-23ebea29ca11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:38.660931 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ffc360-7bc1-4a54-8efc-23ebea29ca11 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:38.667989 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 56/113] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Oct 27 09:23:38 2022] GET /identity/v3/role_assignments?user.id=483ee2e1417c419482b7963c8e6dd62b&scope.project.id=78bd002b14ac4653ae0975911fcabf81&role.id=eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:38.668775 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ffc360-7bc1-4a54-8efc-23ebea29ca11 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:38.765881 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8041107-2d97-4f83-8fd7-00b251cf08c7 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:38.794388 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ffc360-7bc1-4a54-8efc-23ebea29ca11 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:38.803207 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928ce268-d2ba-43ed-ac9c-aa93f739d9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:38.804316 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ce268-d2ba-43ed-ac9c-aa93f739d9d4 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:38.804729 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ce268-d2ba-43ed-ac9c-aa93f739d9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:38.805003 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ce268-d2ba-43ed-ac9c-aa93f739d9d4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:38.806238 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928ce268-d2ba-43ed-ac9c-aa93f739d9d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:38.808240 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928ce268-d2ba-43ed-ac9c-aa93f739d9d4 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:38.814996 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 58/114] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:23:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:38.815776 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928ce268-d2ba-43ed-ac9c-aa93f739d9d4 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:38.822409 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:38.832822 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f092f51-3260-4286-bd5b-f91d6f57eacf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:38.833499 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f092f51-3260-4286-bd5b-f91d6f57eacf None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:38.833782 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f092f51-3260-4286-bd5b-f91d6f57eacf None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:38.834049 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f092f51-3260-4286-bd5b-f91d6f57eacf None admin] PATH_INFO: `/v3/projects/78bd002b14ac4653ae0975911fcabf81/users/22734224a3904550ae5d228ad14143be/roles/f1938eac464c4f22bb492ea2fdf4899a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:38.836060 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f092f51-3260-4286-bd5b-f91d6f57eacf None admin] RBAC: Authorizing `identity:create_grant(project_id=78bd002b14ac4653ae0975911fcabf81, user_id=22734224a3904550ae5d228ad14143be, role_id=f1938eac464c4f22bb492ea2fdf4899a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:38.837482 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f092f51-3260-4286-bd5b-f91d6f57eacf None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:38.849687 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:38.851674 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 57/115] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Oct 27 09:23:38 2022] PUT /identity/v3/projects/78bd002b14ac4653ae0975911fcabf81/users/22734224a3904550ae5d228ad14143be/roles/f1938eac464c4f22bb492ea2fdf4899a => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:23:38.852478 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f092f51-3260-4286-bd5b-f91d6f57eacf None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:38.957528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928ce268-d2ba-43ed-ac9c-aa93f739d9d4 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:38.993233 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f092f51-3260-4286-bd5b-f91d6f57eacf None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:38.999013 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a64013-59a3-43ef-ab02-a37f24292e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:38.999723 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a64013-59a3-43ef-ab02-a37f24292e4c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:38.999936 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a64013-59a3-43ef-ab02-a37f24292e4c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:39.000206 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a64013-59a3-43ef-ab02-a37f24292e4c None admin] PATH_INFO: `/v3/users/483ee2e1417c419482b7963c8e6dd62b` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:39.001963 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a64013-59a3-43ef-ab02-a37f24292e4c None admin] RBAC: Authorizing `identity:get_user(user_id=483ee2e1417c419482b7963c8e6dd62b)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:39.003050 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a64013-59a3-43ef-ab02-a37f24292e4c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:39.009714 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 59/116] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Oct 27 09:23:38 2022] GET /identity/v3/users/483ee2e1417c419482b7963c8e6dd62b => generated 275 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:39.009714 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a64013-59a3-43ef-ab02-a37f24292e4c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:39.038178 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c5a5a-c2e0-4cd2-9201-d7c3bac281cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:39.038831 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c5a5a-c2e0-4cd2-9201-d7c3bac281cf None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:39.039118 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c5a5a-c2e0-4cd2-9201-d7c3bac281cf None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:39.039397 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c5a5a-c2e0-4cd2-9201-d7c3bac281cf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:39.051285 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221c5a5a-c2e0-4cd2-9201-d7c3bac281cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:23:39.057290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-221c5a5a-c2e0-4cd2-9201-d7c3bac281cf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 27 09:23:39.057290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:23:39.057290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:23:39.057290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:23:39.057290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:23:39.057290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:23:39.057290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:23:39.057290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:23:39.057290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:23:39.057290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:23:39.057290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:23:39.057290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:23:39.057290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:23:39.057290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:23:39.057290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:23:39.057290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:23:39.058305 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:23:39.058305 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:23:39.058305 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:23:39.058305 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:23:39.058305 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:23:39.058305 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:23:39.058305 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:23:39.058305 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:23:39.058305 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:23:39.058305 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:23:39.058305 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:23:39.058305 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:23:39.058305 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:23:39.058305 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:23:39.058305 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:23:39.058305 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:23:39.059096 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:23:39.059096 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:23:39.059096 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:23:39.059096 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:23:39.059096 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:23:39.059096 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:23:39.059096 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:23:39.059096 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:23:39.059096 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:23:39.059096 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:23:39.059096 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 27 09:23:39.059096 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:23:39.059096 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 58/117] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Oct 27 09:23:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:23:39.059716 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c5a5a-c2e0-4cd2-9201-d7c3bac281cf None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:39.113525 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:39.117904 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:39.122543 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a64013-59a3-43ef-ab02-a37f24292e4c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:39.159434 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7749ef58-2403-4606-bf83-d3913264e13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:39.160039 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749ef58-2403-4606-bf83-d3913264e13c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:39.160320 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749ef58-2403-4606-bf83-d3913264e13c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:39.160610 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749ef58-2403-4606-bf83-d3913264e13c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:39.173116 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c5a5a-c2e0-4cd2-9201-d7c3bac281cf None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:39.207853 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd882e-1d64-4232-806d-97eebfa82abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:39.208469 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd882e-1d64-4232-806d-97eebfa82abd None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:39.208766 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd882e-1d64-4232-806d-97eebfa82abd None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:39.209039 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd882e-1d64-4232-806d-97eebfa82abd None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:39.209686 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcd882e-1d64-4232-806d-97eebfa82abd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:39.211140 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcd882e-1d64-4232-806d-97eebfa82abd None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:39.238627 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7749ef58-2403-4606-bf83-d3913264e13c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:39.240124 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7749ef58-2403-4606-bf83-d3913264e13c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:39.242697 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 59/118] 10.222.0.70 () {60 vars in 1141 bytes} [Thu Oct 27 09:23:39 2022] POST /identity/v3/projects => generated 302 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:23:39.249523 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd882e-1d64-4232-806d-97eebfa82abd None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:39.382333 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:39.386708 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd882e-1d64-4232-806d-97eebfa82abd None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:39.387200 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:39.395236 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 60/119] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Oct 27 09:23:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 27 09:23:39.396318 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7749ef58-2403-4606-bf83-d3913264e13c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:39.432542 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb3ec6-d283-4be3-af67-83eaded10916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:39.433508 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb3ec6-d283-4be3-af67-83eaded10916 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:39.433508 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb3ec6-d283-4be3-af67-83eaded10916 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:39.433716 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb3ec6-d283-4be3-af67-83eaded10916 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:39.514079 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdb3ec6-d283-4be3-af67-83eaded10916 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:39.515579 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdb3ec6-d283-4be3-af67-83eaded10916 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:39.531640 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7749ef58-2403-4606-bf83-d3913264e13c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:39.573888 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37641244-8037-438a-a09d-5304d4e1c830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:39.574786 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37641244-8037-438a-a09d-5304d4e1c830 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:39.575070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37641244-8037-438a-a09d-5304d4e1c830 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:39.575337 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37641244-8037-438a-a09d-5304d4e1c830 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:39.575995 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37641244-8037-438a-a09d-5304d4e1c830 None admin] RBAC: Adding query filter params (name=member) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:23:39.576295 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37641244-8037-438a-a09d-5304d4e1c830 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:39.577717 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37641244-8037-438a-a09d-5304d4e1c830 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:39.587602 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 61/120] 10.222.0.70 () {56 vars in 1101 bytes} [Thu Oct 27 09:23:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:39.588424 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37641244-8037-438a-a09d-5304d4e1c830 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:39.644723 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:39.651660 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 60/121] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Oct 27 09:23:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 27 09:23:39.652464 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb3ec6-d283-4be3-af67-83eaded10916 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:39.660555 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:39.706933 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37641244-8037-438a-a09d-5304d4e1c830 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:39.747041 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b505c9-885a-4d90-b492-3a311b0a78bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:39.747691 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b505c9-885a-4d90-b492-3a311b0a78bf None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:39.747974 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b505c9-885a-4d90-b492-3a311b0a78bf None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:39.748250 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b505c9-885a-4d90-b492-3a311b0a78bf None admin] PATH_INFO: `/v3/projects/78bd002b14ac4653ae0975911fcabf81` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:39.749433 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b505c9-885a-4d90-b492-3a311b0a78bf None admin] RBAC: Authorizing `identity:get_project(project_id=78bd002b14ac4653ae0975911fcabf81)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:39.750885 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b505c9-885a-4d90-b492-3a311b0a78bf None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:39.752962 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 62/122] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Oct 27 09:23:39 2022] GET /identity/v3/projects/78bd002b14ac4653ae0975911fcabf81 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:39.753530 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b505c9-885a-4d90-b492-3a311b0a78bf None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:39.783295 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb3ec6-d283-4be3-af67-83eaded10916 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:39.818885 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b196c0-96ee-49b8-975f-1fc992589d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:39.819618 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b196c0-96ee-49b8-975f-1fc992589d3b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:39.819967 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b196c0-96ee-49b8-975f-1fc992589d3b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:39.820322 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b196c0-96ee-49b8-975f-1fc992589d3b None admin] PATH_INFO: `/v3/users/483ee2e1417c419482b7963c8e6dd62b` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:39.821552 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b196c0-96ee-49b8-975f-1fc992589d3b None admin] RBAC: Authorizing `identity:get_user(user_id=483ee2e1417c419482b7963c8e6dd62b)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:39.823171 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b196c0-96ee-49b8-975f-1fc992589d3b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:39.824862 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 61/123] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Oct 27 09:23:39 2022] GET /identity/v3/users/483ee2e1417c419482b7963c8e6dd62b => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:39.829249 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b196c0-96ee-49b8-975f-1fc992589d3b None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:39.871030 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b505c9-885a-4d90-b492-3a311b0a78bf None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:39.900918 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:39.903647 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:39.911344 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e04dc-e1e5-44a7-ac75-ab16fa49365d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:39.911980 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e04dc-e1e5-44a7-ac75-ab16fa49365d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:39.912261 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e04dc-e1e5-44a7-ac75-ab16fa49365d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:39.912548 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e04dc-e1e5-44a7-ac75-ab16fa49365d None admin] PATH_INFO: `/v3/projects/78bd002b14ac4653ae0975911fcabf81` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:39.913768 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941e04dc-e1e5-44a7-ac75-ab16fa49365d None admin] RBAC: Authorizing `identity:get_project(project_id=78bd002b14ac4653ae0975911fcabf81)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:39.915274 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941e04dc-e1e5-44a7-ac75-ab16fa49365d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:39.918254 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 63/124] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Oct 27 09:23:39 2022] GET /identity/v3/projects/78bd002b14ac4653ae0975911fcabf81 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:39.919184 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e04dc-e1e5-44a7-ac75-ab16fa49365d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:39.943591 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b196c0-96ee-49b8-975f-1fc992589d3b None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:39.982283 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d871041-798c-4683-82eb-8012f4509d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:39.982905 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d871041-798c-4683-82eb-8012f4509d0d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:39.983189 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d871041-798c-4683-82eb-8012f4509d0d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:39.983463 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d871041-798c-4683-82eb-8012f4509d0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:39.984344 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d871041-798c-4683-82eb-8012f4509d0d None admin] RBAC: Adding query filter params (role.id=0dfd839b7a194aee96f82f4ebc1620a7, scope.project.id=78bd002b14ac4653ae0975911fcabf81, user.id=4e3d5d12b2ba49459e93d0a715783a8a) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:23:39.984705 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d871041-798c-4683-82eb-8012f4509d0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:39.986310 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d871041-798c-4683-82eb-8012f4509d0d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:39.992749 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 62/125] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Oct 27 09:23:39 2022] GET /identity/v3/role_assignments?user.id=4e3d5d12b2ba49459e93d0a715783a8a&scope.project.id=78bd002b14ac4653ae0975911fcabf81&role.id=0dfd839b7a194aee96f82f4ebc1620a7 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:39.993358 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d871041-798c-4683-82eb-8012f4509d0d None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:40.063060 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e04dc-e1e5-44a7-ac75-ab16fa49365d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:40.145337 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cdd8f9-bc81-445a-8d1f-df6dd9b9fc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:40.149463 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cdd8f9-bc81-445a-8d1f-df6dd9b9fc6a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:40.149777 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cdd8f9-bc81-445a-8d1f-df6dd9b9fc6a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:40.150812 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cdd8f9-bc81-445a-8d1f-df6dd9b9fc6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:40.154896 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d871041-798c-4683-82eb-8012f4509d0d None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:40.239186 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:40.245982 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdef85d-b580-4d8b-8c5d-0aaf2eaaa400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:40.246591 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdef85d-b580-4d8b-8c5d-0aaf2eaaa400 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:40.246867 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdef85d-b580-4d8b-8c5d-0aaf2eaaa400 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:40.247133 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdef85d-b580-4d8b-8c5d-0aaf2eaaa400 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:40.247850 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdef85d-b580-4d8b-8c5d-0aaf2eaaa400 None admin] RBAC: Adding query filter params (role.id=fa269b0e4dc54a24bb729398ce38231c, scope.project.id=78bd002b14ac4653ae0975911fcabf81, user.id=483ee2e1417c419482b7963c8e6dd62b) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:23:40.248155 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdef85d-b580-4d8b-8c5d-0aaf2eaaa400 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:40.250451 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdef85d-b580-4d8b-8c5d-0aaf2eaaa400 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:40.250973 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:40.260085 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cdd8f9-bc81-445a-8d1f-df6dd9b9fc6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:40.260722 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 63/126] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Oct 27 09:23:39 2022] GET /identity/v3/role_assignments?user.id=483ee2e1417c419482b7963c8e6dd62b&scope.project.id=78bd002b14ac4653ae0975911fcabf81&role.id=fa269b0e4dc54a24bb729398ce38231c => generated 608 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:40.261717 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cdd8f9-bc81-445a-8d1f-df6dd9b9fc6a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:40.379890 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 64/127] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Oct 27 09:23:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 27 09:23:40.387086 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdef85d-b580-4d8b-8c5d-0aaf2eaaa400 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:40.515972 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdef85d-b580-4d8b-8c5d-0aaf2eaaa400 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:40.551454 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0844306d-20d7-45d5-b919-74fe2b82ba34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:40.552091 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844306d-20d7-45d5-b919-74fe2b82ba34 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:40.552376 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844306d-20d7-45d5-b919-74fe2b82ba34 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:40.552710 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844306d-20d7-45d5-b919-74fe2b82ba34 None admin] PATH_INFO: `/v3/projects/ca239ed7b56748d19640893698c1891c` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:40.553828 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0844306d-20d7-45d5-b919-74fe2b82ba34 None admin] RBAC: Authorizing `identity:get_project(project_id=ca239ed7b56748d19640893698c1891c)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:40.555281 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0844306d-20d7-45d5-b919-74fe2b82ba34 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:40.556977 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 64/128] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Oct 27 09:23:40 2022] GET /identity/v3/projects/ca239ed7b56748d19640893698c1891c => generated 312 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:40.562933 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cdd8f9-bc81-445a-8d1f-df6dd9b9fc6a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:40.660540 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:40.667798 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cdd8f9-bc81-445a-8d1f-df6dd9b9fc6a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:40.702181 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436bf8f2-af66-462e-b3cb-69519b58484b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:40.702790 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436bf8f2-af66-462e-b3cb-69519b58484b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:40.703069 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436bf8f2-af66-462e-b3cb-69519b58484b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:40.703342 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436bf8f2-af66-462e-b3cb-69519b58484b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:40.704058 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436bf8f2-af66-462e-b3cb-69519b58484b None admin] RBAC: Adding query filter params (role.id=eea7cd23e01b4506a9eb6ab86f9adcd2, scope.project.id=ca239ed7b56748d19640893698c1891c, user.id=483ee2e1417c419482b7963c8e6dd62b) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:23:40.704362 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436bf8f2-af66-462e-b3cb-69519b58484b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:40.705816 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436bf8f2-af66-462e-b3cb-69519b58484b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:40.711635 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 65/129] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Oct 27 09:23:40 2022] GET /identity/v3/role_assignments?user.id=483ee2e1417c419482b7963c8e6dd62b&scope.project.id=ca239ed7b56748d19640893698c1891c&role.id=eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:23:41.130167 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf091f1-2d8f-4a73-adeb-d2eb3dded9bd None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:41.130452 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf091f1-2d8f-4a73-adeb-d2eb3dded9bd None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:41.130720 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf091f1-2d8f-4a73-adeb-d2eb3dded9bd None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:41.132059 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 65/130] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:23:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:23:41.137798 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3774b0ca-f5c9-485f-8524-c4c5ac8af004 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:41.138259 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3774b0ca-f5c9-485f-8524-c4c5ac8af004 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:41.138347 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3774b0ca-f5c9-485f-8524-c4c5ac8af004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:41.156894 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-3774b0ca-f5c9-485f-8524-c4c5ac8af004 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:23:41.158080 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3774b0ca-f5c9-485f-8524-c4c5ac8af004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:23:41.223276 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 66/131] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:23:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:23:41.490567 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf091f1-2d8f-4a73-adeb-d2eb3dded9bd None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:41.552233 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:41.574635 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c49bc-0655-4821-9447-895a04cc67d3 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:41.574924 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c49bc-0655-4821-9447-895a04cc67d3 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:41.575202 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c49bc-0655-4821-9447-895a04cc67d3 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:41.576564 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 67/132] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:23:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 27 09:23:41.581278 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1340c400-29de-4c0c-8974-44e6c30b35c6 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:41.581565 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1340c400-29de-4c0c-8974-44e6c30b35c6 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:41.581835 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1340c400-29de-4c0c-8974-44e6c30b35c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:41.592979 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf091f1-2d8f-4a73-adeb-d2eb3dded9bd None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:41.599365 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-1340c400-29de-4c0c-8974-44e6c30b35c6 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:23:41.600578 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1340c400-29de-4c0c-8974-44e6c30b35c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:23:41.627008 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406e84d3-a3f2-420e-9e7c-194d608c70a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:41.627636 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e84d3-a3f2-420e-9e7c-194d608c70a0 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:41.627917 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e84d3-a3f2-420e-9e7c-194d608c70a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:41.628192 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e84d3-a3f2-420e-9e7c-194d608c70a0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:41.638734 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406e84d3-a3f2-420e-9e7c-194d608c70a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:23:41.643556 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-406e84d3-a3f2-420e-9e7c-194d608c70a0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 27 09:23:41.643556 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:23:41.643556 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:23:41.643556 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:23:41.643556 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:23:41.643556 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:23:41.643556 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:23:41.643556 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:23:41.643556 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:23:41.643556 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:23:41.643556 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:23:41.643556 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:23:41.643556 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:23:41.643556 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:23:41.643556 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:23:41.643556 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:23:41.644575 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:23:41.644575 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:23:41.644575 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:23:41.644575 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:23:41.644575 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:23:41.644575 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:23:41.644575 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:23:41.644575 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:23:41.644575 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:23:41.644575 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:23:41.644575 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:23:41.644575 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:23:41.644575 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:23:41.644575 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:23:41.644575 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:23:41.644575 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:23:41.645537 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:23:41.645537 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:23:41.645537 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:23:41.645537 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:23:41.645537 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:23:41.645537 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:23:41.645537 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:23:41.645537 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:23:41.645537 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:23:41.645537 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:23:41.645537 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 27 09:23:41.645537 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:23:41.645537 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 66/133] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Oct 27 09:23:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:23:41.650424 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406e84d3-a3f2-420e-9e7c-194d608c70a0 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:41.668114 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 68/134] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:23:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 27 09:23:41.750637 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406e84d3-a3f2-420e-9e7c-194d608c70a0 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:41.771897 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:41.783166 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cae43a-fd1b-4354-97c6-57d3bcc2c8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:41.783763 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cae43a-fd1b-4354-97c6-57d3bcc2c8b6 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:41.784048 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cae43a-fd1b-4354-97c6-57d3bcc2c8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:41.784320 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cae43a-fd1b-4354-97c6-57d3bcc2c8b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:41.784997 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cae43a-fd1b-4354-97c6-57d3bcc2c8b6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:23:41.785297 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cae43a-fd1b-4354-97c6-57d3bcc2c8b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:41.786660 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cae43a-fd1b-4354-97c6-57d3bcc2c8b6 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:41.794868 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 67/135] 10.222.0.70 () {56 vars in 1101 bytes} [Thu Oct 27 09:23:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:23:41.800493 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1340c400-29de-4c0c-8974-44e6c30b35c6 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:41.823458 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:41.897611 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1340c400-29de-4c0c-8974-44e6c30b35c6 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:41.929629 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20223848-1ee4-47f4-9bfd-5a803d8ff37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:41.930248 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20223848-1ee4-47f4-9bfd-5a803d8ff37c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:41.930527 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20223848-1ee4-47f4-9bfd-5a803d8ff37c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:41.930799 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20223848-1ee4-47f4-9bfd-5a803d8ff37c None admin] PATH_INFO: `/v3/users/22734224a3904550ae5d228ad14143be` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:41.932419 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20223848-1ee4-47f4-9bfd-5a803d8ff37c None admin] RBAC: Authorizing `identity:get_user(user_id=22734224a3904550ae5d228ad14143be)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:41.933868 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20223848-1ee4-47f4-9bfd-5a803d8ff37c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:41.935410 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 69/136] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Oct 27 09:23:41 2022] GET /identity/v3/users/22734224a3904550ae5d228ad14143be => generated 289 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:41.939756 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cae43a-fd1b-4354-97c6-57d3bcc2c8b6 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:41.940609 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20223848-1ee4-47f4-9bfd-5a803d8ff37c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:42.027414 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:42.037234 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cae43a-fd1b-4354-97c6-57d3bcc2c8b6 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:42.039610 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20223848-1ee4-47f4-9bfd-5a803d8ff37c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:42.070673 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5571bd-0b0b-4995-a39f-8703907c6c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:42.071397 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5571bd-0b0b-4995-a39f-8703907c6c6a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:42.071687 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5571bd-0b0b-4995-a39f-8703907c6c6a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:42.071966 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5571bd-0b0b-4995-a39f-8703907c6c6a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:42.073108 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5571bd-0b0b-4995-a39f-8703907c6c6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:42.074234 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26fcdae-d43f-49bb-96c8-fe32df007a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:42.074564 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5571bd-0b0b-4995-a39f-8703907c6c6a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:42.074838 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26fcdae-d43f-49bb-96c8-fe32df007a4c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:42.075117 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26fcdae-d43f-49bb-96c8-fe32df007a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:42.075387 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26fcdae-d43f-49bb-96c8-fe32df007a4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:42.076132 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 68/137] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:23:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:42.081495 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5571bd-0b0b-4995-a39f-8703907c6c6a None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:42.134043 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:42.142251 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26fcdae-d43f-49bb-96c8-fe32df007a4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:42.143709 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26fcdae-d43f-49bb-96c8-fe32df007a4c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:42.183203 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5571bd-0b0b-4995-a39f-8703907c6c6a None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:42.215192 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ed975d-352e-4ac3-93c0-34d1163fd1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:42.215803 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ed975d-352e-4ac3-93c0-34d1163fd1be None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:42.216085 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ed975d-352e-4ac3-93c0-34d1163fd1be None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:42.216362 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ed975d-352e-4ac3-93c0-34d1163fd1be None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:42.217046 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ed975d-352e-4ac3-93c0-34d1163fd1be None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:42.218506 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ed975d-352e-4ac3-93c0-34d1163fd1be None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:42.227978 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:42.241884 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 70/138] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Oct 27 09:23:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 27 09:23:42.247423 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26fcdae-d43f-49bb-96c8-fe32df007a4c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:42.252547 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 69/139] 10.222.0.70 () {60 vars in 1132 bytes} [Thu Oct 27 09:23:42 2022] POST /identity/v3/users => generated 283 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:23:42.359501 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26fcdae-d43f-49bb-96c8-fe32df007a4c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:42.395018 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab99a92d-2a65-4d5d-bc16-94e669583942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:42.395667 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab99a92d-2a65-4d5d-bc16-94e669583942 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:42.395944 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab99a92d-2a65-4d5d-bc16-94e669583942 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:42.396215 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab99a92d-2a65-4d5d-bc16-94e669583942 None admin] PATH_INFO: `/v3/projects/78bd002b14ac4653ae0975911fcabf81` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:42.397407 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab99a92d-2a65-4d5d-bc16-94e669583942 None admin] RBAC: Authorizing `identity:get_project(project_id=78bd002b14ac4653ae0975911fcabf81)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:42.398875 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab99a92d-2a65-4d5d-bc16-94e669583942 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:42.401155 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 71/140] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Oct 27 09:23:42 2022] GET /identity/v3/projects/78bd002b14ac4653ae0975911fcabf81 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:23:42.407293 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ed975d-352e-4ac3-93c0-34d1163fd1be None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:42.510450 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ed975d-352e-4ac3-93c0-34d1163fd1be None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:42.520403 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:42.548734 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1977161d-0619-481c-90f6-2eabc1a182ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:42.549267 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1977161d-0619-481c-90f6-2eabc1a182ef None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:42.549547 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1977161d-0619-481c-90f6-2eabc1a182ef None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:42.549815 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1977161d-0619-481c-90f6-2eabc1a182ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:42.550537 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1977161d-0619-481c-90f6-2eabc1a182ef None admin] RBAC: Adding query filter params (role.id=f1938eac464c4f22bb492ea2fdf4899a, scope.project.id=78bd002b14ac4653ae0975911fcabf81, user.id=22734224a3904550ae5d228ad14143be) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:23:42.550844 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1977161d-0619-481c-90f6-2eabc1a182ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:42.552286 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1977161d-0619-481c-90f6-2eabc1a182ef None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:42.558618 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 70/141] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Oct 27 09:23:42 2022] GET /identity/v3/role_assignments?user.id=22734224a3904550ae5d228ad14143be&scope.project.id=78bd002b14ac4653ae0975911fcabf81&role.id=f1938eac464c4f22bb492ea2fdf4899a => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:44.497159 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7929e4-2ab2-4547-8fb3-65354e0b8831 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:44.497658 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7929e4-2ab2-4547-8fb3-65354e0b8831 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:44.497730 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7929e4-2ab2-4547-8fb3-65354e0b8831 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:44.499069 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 72/142] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:23:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:23:44.504237 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25bbca-1ffa-4d7f-a732-9149d8569793 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:44.504666 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25bbca-1ffa-4d7f-a732-9149d8569793 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:44.504827 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25bbca-1ffa-4d7f-a732-9149d8569793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:44.523467 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-2c25bbca-1ffa-4d7f-a732-9149d8569793 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:23:44.524727 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2c25bbca-1ffa-4d7f-a732-9149d8569793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:23:44.604790 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 71/143] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:23:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:23:44.651903 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d3a035-cb48-497d-af6a-68356b6a32dd None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:44.652196 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d3a035-cb48-497d-af6a-68356b6a32dd None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:44.652458 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d3a035-cb48-497d-af6a-68356b6a32dd None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:44.653973 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 73/144] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:23:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:23:44.658922 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2825d76-5c20-41c0-bc56-9a77a6f39078 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:44.659218 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2825d76-5c20-41c0-bc56-9a77a6f39078 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:44.659496 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2825d76-5c20-41c0-bc56-9a77a6f39078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:44.678188 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-e2825d76-5c20-41c0-bc56-9a77a6f39078 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:23:44.679365 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e2825d76-5c20-41c0-bc56-9a77a6f39078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:23:44.703129 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff14ff-aff6-4e16-b1dd-c2500f3a9ed3 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:44.703419 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff14ff-aff6-4e16-b1dd-c2500f3a9ed3 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:44.703691 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff14ff-aff6-4e16-b1dd-c2500f3a9ed3 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:44.705054 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 74/145] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:23:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 27 09:23:44.709950 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40423b55-bfad-4b03-92ab-6808951cba29 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:44.710234 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40423b55-bfad-4b03-92ab-6808951cba29 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:44.710510 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40423b55-bfad-4b03-92ab-6808951cba29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:44.731454 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-40423b55-bfad-4b03-92ab-6808951cba29 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:23:44.732668 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-40423b55-bfad-4b03-92ab-6808951cba29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:23:44.750916 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 72/146] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:23:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:23:44.785106 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:44.802178 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 75/147] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:23:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:23:44.870451 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2825d76-5c20-41c0-bc56-9a77a6f39078 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:44.905050 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:44.975361 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2825d76-5c20-41c0-bc56-9a77a6f39078 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:45.013941 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7845224-6c5d-4d14-b97c-2ffc777fe7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:45.014567 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7845224-6c5d-4d14-b97c-2ffc777fe7c7 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:45.014840 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7845224-6c5d-4d14-b97c-2ffc777fe7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:45.015104 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7845224-6c5d-4d14-b97c-2ffc777fe7c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:45.025586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7845224-6c5d-4d14-b97c-2ffc777fe7c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:23:45.027210 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40423b55-bfad-4b03-92ab-6808951cba29 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:45.030423 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-f7845224-6c5d-4d14-b97c-2ffc777fe7c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 09:23:45.030423 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:23:45.030423 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:23:45.030423 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:23:45.030423 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:23:45.030423 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:23:45.030423 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:23:45.030423 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:23:45.030423 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:23:45.030423 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:23:45.030423 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:23:45.030423 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:23:45.030423 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:23:45.030423 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:23:45.030423 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:23:45.030423 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:23:45.031280 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:23:45.031280 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:23:45.031280 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:23:45.031280 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:23:45.031280 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:23:45.031280 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:23:45.031280 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:23:45.031280 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:23:45.031280 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:23:45.031280 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:23:45.031280 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:23:45.031280 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:23:45.031280 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:23:45.031280 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:23:45.031280 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:23:45.031280 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:23:45.032383 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:23:45.032383 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:23:45.032383 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:23:45.032383 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:23:45.032383 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:23:45.032383 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:23:45.032383 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:23:45.032383 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:23:45.032383 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:23:45.032383 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:23:45.032383 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 09:23:45.032383 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:23:45.032383 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 73/148] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Oct 27 09:23:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:23:45.036953 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7845224-6c5d-4d14-b97c-2ffc777fe7c7 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:45.129717 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:45.131939 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40423b55-bfad-4b03-92ab-6808951cba29 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:45.139403 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7845224-6c5d-4d14-b97c-2ffc777fe7c7 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:45.166487 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe78bf0-aa37-43bd-8f8a-82b959fb57f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:45.167101 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe78bf0-aa37-43bd-8f8a-82b959fb57f4 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:45.167374 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe78bf0-aa37-43bd-8f8a-82b959fb57f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:45.167664 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe78bf0-aa37-43bd-8f8a-82b959fb57f4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:45.173243 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8167342a-1533-41ab-a47e-4414e704e14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:45.173843 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8167342a-1533-41ab-a47e-4414e704e14e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:45.174121 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8167342a-1533-41ab-a47e-4414e704e14e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:45.174387 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8167342a-1533-41ab-a47e-4414e704e14e None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:45.175037 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8167342a-1533-41ab-a47e-4414e704e14e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:23:45.175337 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8167342a-1533-41ab-a47e-4414e704e14e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:45.176751 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8167342a-1533-41ab-a47e-4414e704e14e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:45.177555 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe78bf0-aa37-43bd-8f8a-82b959fb57f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:23:45.182821 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-cbe78bf0-aa37-43bd-8f8a-82b959fb57f4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 27 09:23:45.182821 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:23:45.182821 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:23:45.182821 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:23:45.182821 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:23:45.182821 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:23:45.182821 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:23:45.182821 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:23:45.182821 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:23:45.182821 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:23:45.182821 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:23:45.182821 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:23:45.182821 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:23:45.182821 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:23:45.182821 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:23:45.182821 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:23:45.183756 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:23:45.183756 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:23:45.183756 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:23:45.183756 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:23:45.183756 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:23:45.183756 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:23:45.183756 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:23:45.183756 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:23:45.183756 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:23:45.183756 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:23:45.183756 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:23:45.183756 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:23:45.183756 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:23:45.183756 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:23:45.183756 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:23:45.183756 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:23:45.184757 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:23:45.184757 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:23:45.184757 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:23:45.184757 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:23:45.184757 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:23:45.184757 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:23:45.184757 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:23:45.184757 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:23:45.184757 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:23:45.184757 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:23:45.184757 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 27 09:23:45.184757 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:23:45.184757 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 76/149] 10.222.0.70 () {56 vars in 1114 bytes} [Thu Oct 27 09:23:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 09:23:45.184757 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe78bf0-aa37-43bd-8f8a-82b959fb57f4 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:45.186310 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 74/150] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Oct 27 09:23:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:23:45.190370 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8167342a-1533-41ab-a47e-4414e704e14e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:45.230153 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:45.286565 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe78bf0-aa37-43bd-8f8a-82b959fb57f4 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:45.288665 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8167342a-1533-41ab-a47e-4414e704e14e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:45.320699 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10843690-863d-4922-ad08-7a5528cd66fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:45.321310 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10843690-863d-4922-ad08-7a5528cd66fa None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:45.321598 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10843690-863d-4922-ad08-7a5528cd66fa None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:45.322433 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10843690-863d-4922-ad08-7a5528cd66fa None admin] PATH_INFO: `/v3/domains/default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:45.322674 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5b9102-50ed-4584-8254-c24ade6345b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:45.323273 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b9102-50ed-4584-8254-c24ade6345b9 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:45.323536 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10843690-863d-4922-ad08-7a5528cd66fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:45.323637 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b9102-50ed-4584-8254-c24ade6345b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:45.323801 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b9102-50ed-4584-8254-c24ade6345b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:45.324446 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5b9102-50ed-4584-8254-c24ade6345b9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:23:45.324776 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5b9102-50ed-4584-8254-c24ade6345b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:45.325011 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10843690-863d-4922-ad08-7a5528cd66fa None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:45.326130 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5b9102-50ed-4584-8254-c24ade6345b9 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:45.326571 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 77/151] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:23:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:45.327353 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10843690-863d-4922-ad08-7a5528cd66fa None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:45.334550 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 75/152] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Oct 27 09:23:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:45.335314 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5b9102-50ed-4584-8254-c24ade6345b9 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:45.352863 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:45.428311 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10843690-863d-4922-ad08-7a5528cd66fa None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:45.432890 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5b9102-50ed-4584-8254-c24ade6345b9 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:45.442539 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:45.463981 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39279b0-f351-4133-af82-e61c991f36c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:45.464639 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39279b0-f351-4133-af82-e61c991f36c3 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:45.464923 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39279b0-f351-4133-af82-e61c991f36c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:45.465203 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39279b0-f351-4133-af82-e61c991f36c3 None admin] PATH_INFO: `/v3/users/a5dd81b1c5c64cd1936e1669bc3e01d7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:45.466735 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014604b4-21ef-4291-b3e4-3bc68063592f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:45.467330 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014604b4-21ef-4291-b3e4-3bc68063592f None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:45.467602 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014604b4-21ef-4291-b3e4-3bc68063592f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:45.467872 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014604b4-21ef-4291-b3e4-3bc68063592f None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:45.468567 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014604b4-21ef-4291-b3e4-3bc68063592f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:45.469996 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014604b4-21ef-4291-b3e4-3bc68063592f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:45.478951 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39279b0-f351-4133-af82-e61c991f36c3 None admin] RBAC: Authorizing `identity:get_user(user_id=a5dd81b1c5c64cd1936e1669bc3e01d7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:45.480727 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39279b0-f351-4133-af82-e61c991f36c3 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:45.482021 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 78/153] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Oct 27 09:23:45 2022] GET /identity/v3/users/a5dd81b1c5c64cd1936e1669bc3e01d7 => generated 283 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:45.482837 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39279b0-f351-4133-af82-e61c991f36c3 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:45.498489 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 76/154] 10.222.0.70 () {60 vars in 1132 bytes} [Thu Oct 27 09:23:45 2022] POST /identity/v3/users => generated 297 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:23:45.499670 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014604b4-21ef-4291-b3e4-3bc68063592f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:45.576199 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:45.610956 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014604b4-21ef-4291-b3e4-3bc68063592f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:45.611205 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39279b0-f351-4133-af82-e61c991f36c3 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:45.646399 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1ace25-ce50-49f7-9168-6d5cf733ec8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:45.647042 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ace25-ce50-49f7-9168-6d5cf733ec8e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:45.647322 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ace25-ce50-49f7-9168-6d5cf733ec8e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:45.647589 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ace25-ce50-49f7-9168-6d5cf733ec8e None admin] PATH_INFO: `/v3/projects/e6e4e23ca6ee4fc0a033668401d7ba30` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:45.648778 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1ace25-ce50-49f7-9168-6d5cf733ec8e None admin] RBAC: Authorizing `identity:get_project(project_id=e6e4e23ca6ee4fc0a033668401d7ba30)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:45.649691 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b746af-eeee-4e6f-a9a4-68d38ba2ed3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:45.650295 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1ace25-ce50-49f7-9168-6d5cf733ec8e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:45.650462 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b746af-eeee-4e6f-a9a4-68d38ba2ed3e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:45.650694 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b746af-eeee-4e6f-a9a4-68d38ba2ed3e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:45.650965 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b746af-eeee-4e6f-a9a4-68d38ba2ed3e None admin] PATH_INFO: `/v3/users/a5dd81b1c5c64cd1936e1669bc3e01d7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:45.657035 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 77/155] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Oct 27 09:23:45 2022] GET /identity/v3/projects/e6e4e23ca6ee4fc0a033668401d7ba30 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:45.657204 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b746af-eeee-4e6f-a9a4-68d38ba2ed3e None admin] RBAC: Authorizing `identity:get_user(user_id=a5dd81b1c5c64cd1936e1669bc3e01d7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:45.657204 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b746af-eeee-4e6f-a9a4-68d38ba2ed3e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:45.657204 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 79/156] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Oct 27 09:23:45 2022] GET /identity/v3/users/a5dd81b1c5c64cd1936e1669bc3e01d7 => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:45.657939 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1ace25-ce50-49f7-9168-6d5cf733ec8e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:45.661678 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b746af-eeee-4e6f-a9a4-68d38ba2ed3e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:45.712365 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:45.782840 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1ace25-ce50-49f7-9168-6d5cf733ec8e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:45.790200 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b746af-eeee-4e6f-a9a4-68d38ba2ed3e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:45.827370 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b973e7-6a32-4a01-83cc-9053bfe4c8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:45.828408 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b973e7-6a32-4a01-83cc-9053bfe4c8fa None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:45.828722 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b973e7-6a32-4a01-83cc-9053bfe4c8fa None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:45.829051 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b973e7-6a32-4a01-83cc-9053bfe4c8fa None admin] PATH_INFO: `/v3/projects/e6e4e23ca6ee4fc0a033668401d7ba30/users/a5dd81b1c5c64cd1936e1669bc3e01d7/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:45.831009 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b973e7-6a32-4a01-83cc-9053bfe4c8fa None admin] RBAC: Authorizing `identity:create_grant(project_id=e6e4e23ca6ee4fc0a033668401d7ba30, user_id=a5dd81b1c5c64cd1936e1669bc3e01d7, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:45.832486 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b973e7-6a32-4a01-83cc-9053bfe4c8fa None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:45.841075 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f10c0a8-97db-4687-a65b-b7f28a214b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:45.841802 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10c0a8-97db-4687-a65b-b7f28a214b62 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:45.842085 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10c0a8-97db-4687-a65b-b7f28a214b62 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:45.842371 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10c0a8-97db-4687-a65b-b7f28a214b62 None admin] PATH_INFO: `/v3/projects/e6e4e23ca6ee4fc0a033668401d7ba30` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:45.843701 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f10c0a8-97db-4687-a65b-b7f28a214b62 None admin] RBAC: Authorizing `identity:get_project(project_id=e6e4e23ca6ee4fc0a033668401d7ba30)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:45.845573 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f10c0a8-97db-4687-a65b-b7f28a214b62 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:45.846773 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:45.847639 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 80/157] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Oct 27 09:23:45 2022] GET /identity/v3/projects/e6e4e23ca6ee4fc0a033668401d7ba30 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:45.851143 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 78/158] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Oct 27 09:23:45 2022] PUT /identity/v3/projects/e6e4e23ca6ee4fc0a033668401d7ba30/users/a5dd81b1c5c64cd1936e1669bc3e01d7/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:23:45.858153 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f10c0a8-97db-4687-a65b-b7f28a214b62 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:45.980040 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f10c0a8-97db-4687-a65b-b7f28a214b62 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:45.991570 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:46.025897 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94a0d0b-dbb3-4379-9538-2756344ed705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:46.026841 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94a0d0b-dbb3-4379-9538-2756344ed705 None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:46.027114 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94a0d0b-dbb3-4379-9538-2756344ed705 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:46.027382 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94a0d0b-dbb3-4379-9538-2756344ed705 None admin] PATH_INFO: `/v3/projects/e6e4e23ca6ee4fc0a033668401d7ba30/users/a5dd81b1c5c64cd1936e1669bc3e01d7/roles/fa269b0e4dc54a24bb729398ce38231c` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:46.029584 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94a0d0b-dbb3-4379-9538-2756344ed705 None admin] RBAC: Authorizing `identity:create_grant(project_id=e6e4e23ca6ee4fc0a033668401d7ba30, user_id=a5dd81b1c5c64cd1936e1669bc3e01d7, role_id=fa269b0e4dc54a24bb729398ce38231c)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:46.031009 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94a0d0b-dbb3-4379-9538-2756344ed705 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:46.045859 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 81/159] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Oct 27 09:23:45 2022] PUT /identity/v3/projects/e6e4e23ca6ee4fc0a033668401d7ba30/users/a5dd81b1c5c64cd1936e1669bc3e01d7/roles/fa269b0e4dc54a24bb729398ce38231c => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:23:47.805874 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee9fe47-d1e0-4e40-8e92-057061389b40 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:47.807029 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee9fe47-d1e0-4e40-8e92-057061389b40 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:47.807029 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee9fe47-d1e0-4e40-8e92-057061389b40 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:47.808611 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 79/160] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:23:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:23:47.814624 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840354e5-2450-4fee-ac5b-298370b7b209 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:47.815597 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840354e5-2450-4fee-ac5b-298370b7b209 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:47.816161 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840354e5-2450-4fee-ac5b-298370b7b209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:47.836081 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-840354e5-2450-4fee-ac5b-298370b7b209 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:23:47.837353 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-840354e5-2450-4fee-ac5b-298370b7b209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:23:47.910210 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 82/161] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:23:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:23:47.924873 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7eeeab-772e-4030-8ca2-7b48a65f85f5 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:47.925483 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7eeeab-772e-4030-8ca2-7b48a65f85f5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:47.925970 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7eeeab-772e-4030-8ca2-7b48a65f85f5 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:47.927877 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 80/162] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:23:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:23:47.933857 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba6cc0-a2a6-44e0-b4ad-5e0956a182b2 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:47.934142 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba6cc0-a2a6-44e0-b4ad-5e0956a182b2 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:47.934417 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba6cc0-a2a6-44e0-b4ad-5e0956a182b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:47.952729 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-1bba6cc0-a2a6-44e0-b4ad-5e0956a182b2 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:23:47.953906 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1bba6cc0-a2a6-44e0-b4ad-5e0956a182b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:23:48.027300 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 83/163] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:23:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:23:48.110075 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037de4d-1c13-4155-bf29-d655213a78a4 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:48.110398 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037de4d-1c13-4155-bf29-d655213a78a4 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:48.110625 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037de4d-1c13-4155-bf29-d655213a78a4 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:48.112099 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 81/164] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:23:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:23:48.118117 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1b6d07-7c41-44e1-9ca5-22766719c902 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:48.118440 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1b6d07-7c41-44e1-9ca5-22766719c902 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:48.118699 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1b6d07-7c41-44e1-9ca5-22766719c902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:48.137374 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-be1b6d07-7c41-44e1-9ca5-22766719c902 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:23:48.138538 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-be1b6d07-7c41-44e1-9ca5-22766719c902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:23:48.162555 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:48.186311 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b037de4d-1c13-4155-bf29-d655213a78a4 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:48.213640 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 84/165] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:23:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:23:48.215021 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd6dc64-0431-4625-a445-34d70ad3b0e7 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:48.215307 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd6dc64-0431-4625-a445-34d70ad3b0e7 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:48.215582 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd6dc64-0431-4625-a445-34d70ad3b0e7 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:48.217595 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 85/166] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:23:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:23:48.223213 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b31670-6ff4-4bac-8578-40a5ac5983f0 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:48.223495 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b31670-6ff4-4bac-8578-40a5ac5983f0 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:48.223766 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b31670-6ff4-4bac-8578-40a5ac5983f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:48.244011 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-05b31670-6ff4-4bac-8578-40a5ac5983f0 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:23:48.245225 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-05b31670-6ff4-4bac-8578-40a5ac5983f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:23:48.322834 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 86/167] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:23:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 27 09:23:48.332411 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b31670-6ff4-4bac-8578-40a5ac5983f0 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:48.335464 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b037de4d-1c13-4155-bf29-d655213a78a4 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:48.381227 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbeb76a-81f9-4df0-981b-f23ef02add97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:48.382213 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbeb76a-81f9-4df0-981b-f23ef02add97 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:48.383006 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbeb76a-81f9-4df0-981b-f23ef02add97 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:48.383451 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbeb76a-81f9-4df0-981b-f23ef02add97 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:48.397192 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbeb76a-81f9-4df0-981b-f23ef02add97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:23:48.404227 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-4cbeb76a-81f9-4df0-981b-f23ef02add97 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 27 09:23:48.404227 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:23:48.404227 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:23:48.404227 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:23:48.404227 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:23:48.404227 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:23:48.404227 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:23:48.404227 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:23:48.404227 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:23:48.404227 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:23:48.404227 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:23:48.404227 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:23:48.404227 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:23:48.404227 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:23:48.404227 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:23:48.404227 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:23:48.405371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:23:48.405371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:23:48.405371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:23:48.405371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:23:48.405371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:23:48.405371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:23:48.405371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:23:48.405371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:23:48.405371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:23:48.405371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:23:48.405371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:23:48.405371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:23:48.405371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:23:48.405371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:23:48.405371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:23:48.405371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:23:48.406350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:23:48.406350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:23:48.406350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:23:48.406350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:23:48.406350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:23:48.406350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:23:48.406350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:23:48.406350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:23:48.406350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:23:48.406350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:23:48.406350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 27 09:23:48.406350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:23:48.408653 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 82/168] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Oct 27 09:23:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 09:23:48.414866 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbeb76a-81f9-4df0-981b-f23ef02add97 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:48.434644 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:48.449436 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b31670-6ff4-4bac-8578-40a5ac5983f0 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:48.485020 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac038c3-3ced-4f09-bbd6-b534aca113bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:48.485745 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac038c3-3ced-4f09-bbd6-b534aca113bb None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:48.486019 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac038c3-3ced-4f09-bbd6-b534aca113bb None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:48.486285 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac038c3-3ced-4f09-bbd6-b534aca113bb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:48.487458 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac038c3-3ced-4f09-bbd6-b534aca113bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:48.488987 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac038c3-3ced-4f09-bbd6-b534aca113bb None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:48.490540 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 87/169] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:23:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:23:48.492498 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:48.496059 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac038c3-3ced-4f09-bbd6-b534aca113bb None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:48.537986 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbeb76a-81f9-4df0-981b-f23ef02add97 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:48.572797 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee608340-b06c-49ef-8984-3c71aefda7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:48.573396 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee608340-b06c-49ef-8984-3c71aefda7ed None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:48.573790 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee608340-b06c-49ef-8984-3c71aefda7ed None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:48.574062 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee608340-b06c-49ef-8984-3c71aefda7ed None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:48.574716 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee608340-b06c-49ef-8984-3c71aefda7ed None admin] RBAC: Adding query filter params (name=member) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:23:48.575013 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee608340-b06c-49ef-8984-3c71aefda7ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:48.576583 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee608340-b06c-49ef-8984-3c71aefda7ed None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:48.588630 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 83/170] 10.222.0.70 () {56 vars in 1101 bytes} [Thu Oct 27 09:23:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:48.588630 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee608340-b06c-49ef-8984-3c71aefda7ed None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:48.601151 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac038c3-3ced-4f09-bbd6-b534aca113bb None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:48.637638 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc858283-5a64-48d7-9148-69f5ef745711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:48.638193 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc858283-5a64-48d7-9148-69f5ef745711 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:48.638473 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc858283-5a64-48d7-9148-69f5ef745711 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:48.638741 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc858283-5a64-48d7-9148-69f5ef745711 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:48.648797 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc858283-5a64-48d7-9148-69f5ef745711 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:23:48.654183 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-fc858283-5a64-48d7-9148-69f5ef745711 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 09:23:48.654183 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:23:48.654183 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:23:48.654183 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:23:48.654183 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:23:48.654183 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:23:48.654183 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:23:48.654183 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:23:48.654183 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:23:48.654183 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:23:48.654183 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:23:48.654183 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:23:48.654183 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:23:48.654183 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:23:48.654183 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:23:48.654183 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:23:48.655157 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:23:48.655157 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:23:48.655157 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:23:48.655157 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:23:48.655157 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:23:48.655157 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:23:48.655157 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:23:48.655157 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:23:48.655157 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:23:48.655157 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:23:48.655157 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:23:48.655157 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:23:48.655157 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:23:48.655157 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:23:48.655157 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:23:48.655157 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:23:48.655924 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:23:48.655924 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:23:48.655924 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:23:48.655924 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:23:48.655924 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:23:48.655924 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:23:48.655924 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:23:48.655924 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:23:48.655924 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:23:48.655924 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:23:48.655924 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 09:23:48.655924 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:23:48.655924 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 88/171] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Oct 27 09:23:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:23:48.656593 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc858283-5a64-48d7-9148-69f5ef745711 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:48.674738 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:48.691723 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee608340-b06c-49ef-8984-3c71aefda7ed None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:48.714732 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:48.726753 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf7655e-8804-45cf-8026-988bfc307a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:48.727363 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7655e-8804-45cf-8026-988bfc307a18 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:48.727647 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7655e-8804-45cf-8026-988bfc307a18 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:48.727918 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7655e-8804-45cf-8026-988bfc307a18 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:48.728595 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf7655e-8804-45cf-8026-988bfc307a18 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:48.730035 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf7655e-8804-45cf-8026-988bfc307a18 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:48.750658 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 84/172] 10.222.0.70 () {60 vars in 1132 bytes} [Thu Oct 27 09:23:48 2022] POST /identity/v3/users => generated 297 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:23:48.751465 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf7655e-8804-45cf-8026-988bfc307a18 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:48.757693 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc858283-5a64-48d7-9148-69f5ef745711 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:48.798454 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c60d750-7c2b-4dd2-9223-f8dc5b3f2823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:48.799096 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c60d750-7c2b-4dd2-9223-f8dc5b3f2823 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:48.799377 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c60d750-7c2b-4dd2-9223-f8dc5b3f2823 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:48.799649 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c60d750-7c2b-4dd2-9223-f8dc5b3f2823 None admin] PATH_INFO: `/v3/users/de77771d93024bfdb51885538564c97a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:48.815426 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c60d750-7c2b-4dd2-9223-f8dc5b3f2823 None admin] RBAC: Authorizing `identity:get_user(user_id=de77771d93024bfdb51885538564c97a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:48.816949 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c60d750-7c2b-4dd2-9223-f8dc5b3f2823 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:48.818829 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 89/173] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Oct 27 09:23:48 2022] GET /identity/v3/users/de77771d93024bfdb51885538564c97a => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:48.824984 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c60d750-7c2b-4dd2-9223-f8dc5b3f2823 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:48.880484 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf7655e-8804-45cf-8026-988bfc307a18 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:48.910666 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:48.918839 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f75e3f-4484-4f1c-80ab-d23fcef1927c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:48.919481 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f75e3f-4484-4f1c-80ab-d23fcef1927c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:48.919769 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f75e3f-4484-4f1c-80ab-d23fcef1927c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:48.920040 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f75e3f-4484-4f1c-80ab-d23fcef1927c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:48.933038 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f75e3f-4484-4f1c-80ab-d23fcef1927c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:23:48.937784 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-d6f75e3f-4484-4f1c-80ab-d23fcef1927c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 27 09:23:48.937784 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:23:48.937784 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:23:48.937784 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:23:48.937784 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:23:48.937784 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:23:48.937784 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:23:48.937784 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:23:48.937784 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:23:48.937784 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:23:48.937784 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:23:48.937784 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:23:48.937784 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:23:48.937784 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:23:48.937784 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:23:48.937784 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:23:48.938667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:23:48.938667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:23:48.938667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:23:48.938667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:23:48.938667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:23:48.938667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:23:48.938667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:23:48.938667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:23:48.938667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:23:48.938667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:23:48.938667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:23:48.938667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:23:48.938667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:23:48.938667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:23:48.938667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:23:48.938667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:23:48.941243 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:23:48.941243 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:23:48.941243 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:23:48.941243 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:23:48.941243 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:23:48.941243 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:23:48.941243 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:23:48.941243 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:23:48.941243 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:23:48.941243 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:23:48.941243 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 27 09:23:48.941243 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:23:48.941243 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 85/174] 10.222.0.70 () {56 vars in 1114 bytes} [Thu Oct 27 09:23:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:23:48.941243 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f75e3f-4484-4f1c-80ab-d23fcef1927c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:48.947976 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c60d750-7c2b-4dd2-9223-f8dc5b3f2823 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:48.986183 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:48.989041 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51da920-dbab-432d-8cf5-908e4817e36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:48.989907 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51da920-dbab-432d-8cf5-908e4817e36b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:48.990506 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51da920-dbab-432d-8cf5-908e4817e36b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:48.990798 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51da920-dbab-432d-8cf5-908e4817e36b None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:48.991485 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51da920-dbab-432d-8cf5-908e4817e36b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:23:48.991859 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51da920-dbab-432d-8cf5-908e4817e36b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:48.993259 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51da920-dbab-432d-8cf5-908e4817e36b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:49.003760 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 90/175] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Oct 27 09:23:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:49.003760 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51da920-dbab-432d-8cf5-908e4817e36b None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:49.073621 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f75e3f-4484-4f1c-80ab-d23fcef1927c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:49.120764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30172d44-67ac-46f6-a61c-3968d3a91f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:49.121394 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30172d44-67ac-46f6-a61c-3968d3a91f9d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:49.121671 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30172d44-67ac-46f6-a61c-3968d3a91f9d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:49.121936 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30172d44-67ac-46f6-a61c-3968d3a91f9d None admin] PATH_INFO: `/v3/projects/e6e4e23ca6ee4fc0a033668401d7ba30` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:49.123145 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30172d44-67ac-46f6-a61c-3968d3a91f9d None admin] RBAC: Authorizing `identity:get_project(project_id=e6e4e23ca6ee4fc0a033668401d7ba30)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:49.124670 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30172d44-67ac-46f6-a61c-3968d3a91f9d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:49.126308 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 86/176] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Oct 27 09:23:48 2022] GET /identity/v3/projects/e6e4e23ca6ee4fc0a033668401d7ba30 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:49.128271 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30172d44-67ac-46f6-a61c-3968d3a91f9d None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:49.147598 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51da920-dbab-432d-8cf5-908e4817e36b None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:49.182571 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e81164-8bdb-4713-b95a-f289618e5e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:49.183419 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e81164-8bdb-4713-b95a-f289618e5e30 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:49.183700 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e81164-8bdb-4713-b95a-f289618e5e30 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:49.183971 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e81164-8bdb-4713-b95a-f289618e5e30 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:49.184681 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e81164-8bdb-4713-b95a-f289618e5e30 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:23:49.184990 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e81164-8bdb-4713-b95a-f289618e5e30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:49.186365 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e81164-8bdb-4713-b95a-f289618e5e30 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:49.188882 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:49.202370 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 91/177] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Oct 27 09:23:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:49.203187 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e81164-8bdb-4713-b95a-f289618e5e30 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:49.265507 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30172d44-67ac-46f6-a61c-3968d3a91f9d None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:49.276018 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:49.309507 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b3d03-048e-422d-aed6-3ec53cbbc3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:49.310235 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b3d03-048e-422d-aed6-3ec53cbbc3de None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:49.310546 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b3d03-048e-422d-aed6-3ec53cbbc3de None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:49.310871 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b3d03-048e-422d-aed6-3ec53cbbc3de None admin] PATH_INFO: `/v3/users/a5dd81b1c5c64cd1936e1669bc3e01d7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:49.312036 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b3d03-048e-422d-aed6-3ec53cbbc3de None admin] RBAC: Authorizing `identity:get_user(user_id=a5dd81b1c5c64cd1936e1669bc3e01d7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:49.313512 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b3d03-048e-422d-aed6-3ec53cbbc3de None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:49.315275 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 87/178] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Oct 27 09:23:49 2022] GET /identity/v3/users/a5dd81b1c5c64cd1936e1669bc3e01d7 => generated 283 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:49.315859 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b3d03-048e-422d-aed6-3ec53cbbc3de None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:49.340087 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e81164-8bdb-4713-b95a-f289618e5e30 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:49.387727 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf962c0-0152-4a17-a018-3d05f4b1487d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:49.388681 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf962c0-0152-4a17-a018-3d05f4b1487d None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:49.389203 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf962c0-0152-4a17-a018-3d05f4b1487d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:49.390256 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf962c0-0152-4a17-a018-3d05f4b1487d None admin] PATH_INFO: `/v3/projects/e6e4e23ca6ee4fc0a033668401d7ba30/users/de77771d93024bfdb51885538564c97a/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:49.393031 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf962c0-0152-4a17-a018-3d05f4b1487d None admin] RBAC: Authorizing `identity:create_grant(project_id=e6e4e23ca6ee4fc0a033668401d7ba30, user_id=de77771d93024bfdb51885538564c97a, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:49.395558 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf962c0-0152-4a17-a018-3d05f4b1487d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:49.415106 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 92/179] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Oct 27 09:23:49 2022] PUT /identity/v3/projects/e6e4e23ca6ee4fc0a033668401d7ba30/users/de77771d93024bfdb51885538564c97a/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:23:49.416973 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf962c0-0152-4a17-a018-3d05f4b1487d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:49.431136 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b3d03-048e-422d-aed6-3ec53cbbc3de None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:49.472068 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a44e1-a1c3-4bac-8ef3-6962a4bfd050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:49.472673 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a44e1-a1c3-4bac-8ef3-6962a4bfd050 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:49.472953 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a44e1-a1c3-4bac-8ef3-6962a4bfd050 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:49.473227 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a44e1-a1c3-4bac-8ef3-6962a4bfd050 None admin] PATH_INFO: `/v3/users/a5dd81b1c5c64cd1936e1669bc3e01d7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:49.474517 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946a44e1-a1c3-4bac-8ef3-6962a4bfd050 None admin] RBAC: Authorizing `identity:get_user(user_id=a5dd81b1c5c64cd1936e1669bc3e01d7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:49.476063 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946a44e1-a1c3-4bac-8ef3-6962a4bfd050 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:49.478255 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 88/180] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Oct 27 09:23:49 2022] GET /identity/v3/users/a5dd81b1c5c64cd1936e1669bc3e01d7 => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:49.485866 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a44e1-a1c3-4bac-8ef3-6962a4bfd050 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:49.498889 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:49.551695 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf962c0-0152-4a17-a018-3d05f4b1487d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:49.583649 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:49.598454 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e9619e-d0e6-4afa-9c47-9cd4b0331545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:49.599451 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e9619e-d0e6-4afa-9c47-9cd4b0331545 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:49.600451 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e9619e-d0e6-4afa-9c47-9cd4b0331545 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:49.600978 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e9619e-d0e6-4afa-9c47-9cd4b0331545 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:49.675137 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a44e1-a1c3-4bac-8ef3-6962a4bfd050 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:49.683750 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e9619e-d0e6-4afa-9c47-9cd4b0331545 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:49.685566 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e9619e-d0e6-4afa-9c47-9cd4b0331545 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:49.717923 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3100f1b-e085-4b73-be41-e3ae925677a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:49.718725 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3100f1b-e085-4b73-be41-e3ae925677a1 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:49.719106 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3100f1b-e085-4b73-be41-e3ae925677a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:49.719476 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3100f1b-e085-4b73-be41-e3ae925677a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:49.751752 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:49.802567 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 93/181] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Oct 27 09:23:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 27 09:23:49.808985 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e9619e-d0e6-4afa-9c47-9cd4b0331545 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:49.843594 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3100f1b-e085-4b73-be41-e3ae925677a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:49.848628 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3100f1b-e085-4b73-be41-e3ae925677a1 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:49.938721 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:49.976588 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e9619e-d0e6-4afa-9c47-9cd4b0331545 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:50.037907 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 89/182] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Oct 27 09:23:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 27 09:23:50.045941 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3100f1b-e085-4b73-be41-e3ae925677a1 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:50.047646 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa11bfd-8266-4d71-a3be-3dd714d64fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:50.048535 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa11bfd-8266-4d71-a3be-3dd714d64fb8 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:50.049012 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa11bfd-8266-4d71-a3be-3dd714d64fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:50.050383 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa11bfd-8266-4d71-a3be-3dd714d64fb8 None admin] PATH_INFO: `/v3/projects/e6e4e23ca6ee4fc0a033668401d7ba30` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:50.053529 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa11bfd-8266-4d71-a3be-3dd714d64fb8 None admin] RBAC: Authorizing `identity:get_project(project_id=e6e4e23ca6ee4fc0a033668401d7ba30)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:50.058022 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa11bfd-8266-4d71-a3be-3dd714d64fb8 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:50.061755 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 94/183] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Oct 27 09:23:49 2022] GET /identity/v3/projects/e6e4e23ca6ee4fc0a033668401d7ba30 => generated 302 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:50.067586 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa11bfd-8266-4d71-a3be-3dd714d64fb8 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:50.096760 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:50.185345 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa11bfd-8266-4d71-a3be-3dd714d64fb8 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:50.245362 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3100f1b-e085-4b73-be41-e3ae925677a1 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:50.251393 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc9d77-2544-408b-b9aa-6178f36eb9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:50.252010 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc9d77-2544-408b-b9aa-6178f36eb9b2 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:50.252285 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc9d77-2544-408b-b9aa-6178f36eb9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:50.252573 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc9d77-2544-408b-b9aa-6178f36eb9b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:50.253295 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc9d77-2544-408b-b9aa-6178f36eb9b2 None admin] RBAC: Adding query filter params (role.id=0dfd839b7a194aee96f82f4ebc1620a7, scope.project.id=e6e4e23ca6ee4fc0a033668401d7ba30, user.id=a5dd81b1c5c64cd1936e1669bc3e01d7) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:23:50.253599 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc9d77-2544-408b-b9aa-6178f36eb9b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:50.255037 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc9d77-2544-408b-b9aa-6178f36eb9b2 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:50.271608 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 95/184] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Oct 27 09:23:50 2022] GET /identity/v3/role_assignments?user.id=a5dd81b1c5c64cd1936e1669bc3e01d7&scope.project.id=e6e4e23ca6ee4fc0a033668401d7ba30&role.id=0dfd839b7a194aee96f82f4ebc1620a7 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:23:50.293722 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9fdf58-e188-4729-ac8b-8a7e6b4f182d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:50.294296 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9fdf58-e188-4729-ac8b-8a7e6b4f182d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:50.294577 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9fdf58-e188-4729-ac8b-8a7e6b4f182d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:50.294848 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9fdf58-e188-4729-ac8b-8a7e6b4f182d None admin] PATH_INFO: `/v3/projects/e6e4e23ca6ee4fc0a033668401d7ba30` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:50.296065 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9fdf58-e188-4729-ac8b-8a7e6b4f182d None admin] RBAC: Authorizing `identity:get_project(project_id=e6e4e23ca6ee4fc0a033668401d7ba30)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:50.297613 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9fdf58-e188-4729-ac8b-8a7e6b4f182d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:50.299551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 90/185] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Oct 27 09:23:50 2022] GET /identity/v3/projects/e6e4e23ca6ee4fc0a033668401d7ba30 => generated 302 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:50.306548 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc9d77-2544-408b-b9aa-6178f36eb9b2 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:50.410854 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:50.427640 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc9d77-2544-408b-b9aa-6178f36eb9b2 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:50.468223 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d36eb-0960-4160-9f2e-d150f5b6ada1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:50.468886 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d36eb-0960-4160-9f2e-d150f5b6ada1 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:50.469194 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d36eb-0960-4160-9f2e-d150f5b6ada1 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:50.469437 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d36eb-0960-4160-9f2e-d150f5b6ada1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:50.470179 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578d36eb-0960-4160-9f2e-d150f5b6ada1 None admin] RBAC: Adding query filter params (role.id=fa269b0e4dc54a24bb729398ce38231c, scope.project.id=e6e4e23ca6ee4fc0a033668401d7ba30, user.id=a5dd81b1c5c64cd1936e1669bc3e01d7) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:23:50.470755 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578d36eb-0960-4160-9f2e-d150f5b6ada1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:50.472225 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578d36eb-0960-4160-9f2e-d150f5b6ada1 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:50.479883 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 96/186] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Oct 27 09:23:50 2022] GET /identity/v3/role_assignments?user.id=a5dd81b1c5c64cd1936e1669bc3e01d7&scope.project.id=e6e4e23ca6ee4fc0a033668401d7ba30&role.id=fa269b0e4dc54a24bb729398ce38231c => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:51.233170 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e712814-c591-4548-b1f7-eac135662a69 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:51.233683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e712814-c591-4548-b1f7-eac135662a69 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:51.233683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e712814-c591-4548-b1f7-eac135662a69 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:51.235197 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 91/187] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:23:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:23:51.240251 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657733e-e700-4e91-97d4-f161eca309f5 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:51.240665 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657733e-e700-4e91-97d4-f161eca309f5 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:51.240820 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657733e-e700-4e91-97d4-f161eca309f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:51.260133 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-b657733e-e700-4e91-97d4-f161eca309f5 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:23:51.261328 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b657733e-e700-4e91-97d4-f161eca309f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:23:51.334694 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 97/188] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:23:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:23:51.443468 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba750e5-ebe2-40b0-8577-405a8138af5f None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:51.443751 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba750e5-ebe2-40b0-8577-405a8138af5f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:51.444199 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba750e5-ebe2-40b0-8577-405a8138af5f None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:51.445576 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 92/189] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:23:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:23:51.451333 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5b81e-121d-499b-98fa-83113bdbae38 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:51.451624 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5b81e-121d-499b-98fa-83113bdbae38 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:51.451901 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5b81e-121d-499b-98fa-83113bdbae38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:51.470047 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-00e5b81e-121d-499b-98fa-83113bdbae38 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:23:51.471480 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-00e5b81e-121d-499b-98fa-83113bdbae38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:23:51.538314 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 98/190] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:23:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:23:51.603604 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba750e5-ebe2-40b0-8577-405a8138af5f None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:51.616771 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:51.710204 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba750e5-ebe2-40b0-8577-405a8138af5f None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:51.745609 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9825256-7ca5-4ede-813a-e80b1eca3eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:51.746339 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9825256-7ca5-4ede-813a-e80b1eca3eb2 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:51.746625 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9825256-7ca5-4ede-813a-e80b1eca3eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:51.746896 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9825256-7ca5-4ede-813a-e80b1eca3eb2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:51.748060 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9825256-7ca5-4ede-813a-e80b1eca3eb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:51.749667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9825256-7ca5-4ede-813a-e80b1eca3eb2 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:51.751545 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 93/191] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:23:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:51.757140 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e5b81e-121d-499b-98fa-83113bdbae38 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:51.774377 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:51.808069 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9825256-7ca5-4ede-813a-e80b1eca3eb2 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:51.867754 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e5b81e-121d-499b-98fa-83113bdbae38 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:51.892958 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:51.905160 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc70f1b-34af-49b3-af1e-5000cf1cf4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:51.905772 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc70f1b-34af-49b3-af1e-5000cf1cf4a3 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:51.906047 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc70f1b-34af-49b3-af1e-5000cf1cf4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:51.906315 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc70f1b-34af-49b3-af1e-5000cf1cf4a3 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:51.906960 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc70f1b-34af-49b3-af1e-5000cf1cf4a3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:51.908383 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc70f1b-34af-49b3-af1e-5000cf1cf4a3 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:51.914834 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9825256-7ca5-4ede-813a-e80b1eca3eb2 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:51.930268 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 99/192] 10.222.0.70 () {60 vars in 1132 bytes} [Thu Oct 27 09:23:51 2022] POST /identity/v3/users => generated 293 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:23:51.931592 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d9f07-0348-4bde-859d-6433cc0e1588 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:51.931943 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d9f07-0348-4bde-859d-6433cc0e1588 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:51.932498 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d9f07-0348-4bde-859d-6433cc0e1588 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:51.933608 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 100/193] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:23:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:23:51.942320 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8af2e-b451-4be0-a765-4b7c0a91cdce None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:51.942597 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8af2e-b451-4be0-a765-4b7c0a91cdce None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:51.942874 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8af2e-b451-4be0-a765-4b7c0a91cdce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:51.962964 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-d4d8af2e-b451-4be0-a765-4b7c0a91cdce None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:23:51.964183 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d4d8af2e-b451-4be0-a765-4b7c0a91cdce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:23:51.965076 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d51b18-79a8-4cf9-abd6-adc68b6d9e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:51.965725 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d51b18-79a8-4cf9-abd6-adc68b6d9e62 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:51.965996 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d51b18-79a8-4cf9-abd6-adc68b6d9e62 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:51.966264 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d51b18-79a8-4cf9-abd6-adc68b6d9e62 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:51.979000 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d51b18-79a8-4cf9-abd6-adc68b6d9e62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:23:51.984447 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-a2d51b18-79a8-4cf9-abd6-adc68b6d9e62 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 27 09:23:51.984447 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:23:51.984447 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:23:51.984447 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:23:51.984447 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:23:51.984447 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:23:51.984447 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:23:51.984447 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:23:51.984447 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:23:51.984447 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:23:51.984447 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:23:51.984447 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:23:51.984447 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:23:51.984447 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:23:51.984447 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:23:51.984447 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:23:51.985440 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:23:51.985440 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:23:51.985440 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:23:51.985440 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:23:51.985440 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:23:51.985440 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:23:51.985440 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:23:51.985440 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:23:51.985440 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:23:51.985440 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:23:51.985440 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:23:51.985440 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:23:51.985440 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:23:51.985440 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:23:51.985440 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:23:51.985440 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:23:51.986229 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:23:51.986229 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:23:51.986229 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:23:51.986229 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:23:51.986229 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:23:51.986229 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:23:51.986229 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:23:51.986229 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:23:51.986229 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:23:51.986229 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:23:51.986229 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 27 09:23:51.986229 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:23:51.986229 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 94/194] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Oct 27 09:23:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 09:23:51.991893 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d51b18-79a8-4cf9-abd6-adc68b6d9e62 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:52.044800 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:52.052153 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 101/195] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:23:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:23:52.116661 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d51b18-79a8-4cf9-abd6-adc68b6d9e62 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:52.150957 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e97eb-f85b-4e63-8865-0221a0ba85db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:52.151554 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e97eb-f85b-4e63-8865-0221a0ba85db None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:52.151828 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e97eb-f85b-4e63-8865-0221a0ba85db None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:52.152103 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e97eb-f85b-4e63-8865-0221a0ba85db None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:52.152778 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197e97eb-f85b-4e63-8865-0221a0ba85db None admin] RBAC: Adding query filter params (name=reader) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:23:52.153072 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197e97eb-f85b-4e63-8865-0221a0ba85db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:52.154439 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197e97eb-f85b-4e63-8865-0221a0ba85db None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:52.159925 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:52.168862 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 95/196] 10.222.0.70 () {56 vars in 1101 bytes} [Thu Oct 27 09:23:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:23:52.170877 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d8af2e-b451-4be0-a765-4b7c0a91cdce None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:52.279833 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d8af2e-b451-4be0-a765-4b7c0a91cdce None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:52.320484 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1d88ab-6da6-497a-8efb-619897534691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:52.321147 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d88ab-6da6-497a-8efb-619897534691 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:52.321945 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d88ab-6da6-497a-8efb-619897534691 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:52.322217 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d88ab-6da6-497a-8efb-619897534691 None admin] PATH_INFO: `/v3/users/41179171b7644ed8a51bc41ab1678469` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:52.325528 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e97eb-f85b-4e63-8865-0221a0ba85db None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:52.339043 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1d88ab-6da6-497a-8efb-619897534691 None admin] RBAC: Authorizing `identity:get_user(user_id=41179171b7644ed8a51bc41ab1678469)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:52.340510 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1d88ab-6da6-497a-8efb-619897534691 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:52.342211 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 102/197] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Oct 27 09:23:52 2022] GET /identity/v3/users/41179171b7644ed8a51bc41ab1678469 => generated 297 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:52.347996 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1d88ab-6da6-497a-8efb-619897534691 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:52.422319 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:52.448392 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e97eb-f85b-4e63-8865-0221a0ba85db None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:52.475110 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1d88ab-6da6-497a-8efb-619897534691 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:52.486380 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e119f-1cac-4e9c-b72d-1de2fc3ac3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:52.487129 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e119f-1cac-4e9c-b72d-1de2fc3ac3cf None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:52.487476 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e119f-1cac-4e9c-b72d-1de2fc3ac3cf None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:52.487817 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e119f-1cac-4e9c-b72d-1de2fc3ac3cf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:52.499362 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69e119f-1cac-4e9c-b72d-1de2fc3ac3cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:23:52.504754 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-c69e119f-1cac-4e9c-b72d-1de2fc3ac3cf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 27 09:23:52.504754 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:23:52.504754 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:23:52.504754 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:23:52.504754 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:23:52.504754 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:23:52.504754 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:23:52.504754 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:23:52.504754 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:23:52.504754 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:23:52.504754 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:23:52.504754 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:23:52.504754 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:23:52.504754 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:23:52.504754 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:23:52.504754 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:23:52.505806 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:23:52.505806 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:23:52.505806 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:23:52.505806 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:23:52.505806 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:23:52.505806 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:23:52.505806 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:23:52.505806 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:23:52.505806 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:23:52.505806 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:23:52.505806 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:23:52.505806 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:23:52.505806 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:23:52.505806 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:23:52.505806 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:23:52.505806 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:23:52.506793 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:23:52.506793 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:23:52.506793 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:23:52.506793 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:23:52.506793 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:23:52.506793 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:23:52.506793 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:23:52.506793 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:23:52.506793 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:23:52.506793 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:23:52.506793 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 27 09:23:52.506793 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:23:52.508585 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 96/198] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Oct 27 09:23:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 09:23:52.514198 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e119f-1cac-4e9c-b72d-1de2fc3ac3cf None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:52.531348 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f491c252-0541-4eb5-a61d-4627e571a4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:52.532417 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f491c252-0541-4eb5-a61d-4627e571a4f4 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:52.533036 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f491c252-0541-4eb5-a61d-4627e571a4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:52.533505 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f491c252-0541-4eb5-a61d-4627e571a4f4 None admin] PATH_INFO: `/v3/projects/e6e4e23ca6ee4fc0a033668401d7ba30` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:52.534838 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f491c252-0541-4eb5-a61d-4627e571a4f4 None admin] RBAC: Authorizing `identity:get_project(project_id=e6e4e23ca6ee4fc0a033668401d7ba30)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:52.537193 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f491c252-0541-4eb5-a61d-4627e571a4f4 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:52.539065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 103/199] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Oct 27 09:23:52 2022] GET /identity/v3/projects/e6e4e23ca6ee4fc0a033668401d7ba30 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:23:52.547955 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f491c252-0541-4eb5-a61d-4627e571a4f4 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:52.581961 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:52.627269 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e119f-1cac-4e9c-b72d-1de2fc3ac3cf None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:52.663003 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bf2d54-c052-4130-aa70-703243afe17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:52.663651 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf2d54-c052-4130-aa70-703243afe17f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:52.663933 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf2d54-c052-4130-aa70-703243afe17f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:52.664206 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf2d54-c052-4130-aa70-703243afe17f None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:52.664913 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bf2d54-c052-4130-aa70-703243afe17f None admin] RBAC: Adding query filter params (name=member) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:23:52.665251 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bf2d54-c052-4130-aa70-703243afe17f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:52.666649 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bf2d54-c052-4130-aa70-703243afe17f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:52.671370 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f491c252-0541-4eb5-a61d-4627e571a4f4 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:52.675461 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 97/200] 10.222.0.70 () {56 vars in 1101 bytes} [Thu Oct 27 09:23:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:52.681430 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bf2d54-c052-4130-aa70-703243afe17f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:52.703483 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:52.714810 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6720ce1d-7bcc-4a7f-8fd0-a1473ec2c1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:52.715631 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6720ce1d-7bcc-4a7f-8fd0-a1473ec2c1d4 None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:52.716218 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6720ce1d-7bcc-4a7f-8fd0-a1473ec2c1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:52.716500 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6720ce1d-7bcc-4a7f-8fd0-a1473ec2c1d4 None admin] PATH_INFO: `/v3/projects/e6e4e23ca6ee4fc0a033668401d7ba30/users/41179171b7644ed8a51bc41ab1678469/roles/f1938eac464c4f22bb492ea2fdf4899a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:52.718735 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6720ce1d-7bcc-4a7f-8fd0-a1473ec2c1d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=e6e4e23ca6ee4fc0a033668401d7ba30, user_id=41179171b7644ed8a51bc41ab1678469, role_id=f1938eac464c4f22bb492ea2fdf4899a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:52.721264 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6720ce1d-7bcc-4a7f-8fd0-a1473ec2c1d4 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:52.740360 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 104/201] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Oct 27 09:23:52 2022] PUT /identity/v3/projects/e6e4e23ca6ee4fc0a033668401d7ba30/users/41179171b7644ed8a51bc41ab1678469/roles/f1938eac464c4f22bb492ea2fdf4899a => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:23:52.804798 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bf2d54-c052-4130-aa70-703243afe17f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:52.832089 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:52.847638 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead3ddcb-7d1b-4b57-a0eb-97eec0dbe7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:52.848292 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead3ddcb-7d1b-4b57-a0eb-97eec0dbe7fd None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:52.848663 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead3ddcb-7d1b-4b57-a0eb-97eec0dbe7fd None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:52.848935 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead3ddcb-7d1b-4b57-a0eb-97eec0dbe7fd None admin] PATH_INFO: `/v3/users/de77771d93024bfdb51885538564c97a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:52.850187 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead3ddcb-7d1b-4b57-a0eb-97eec0dbe7fd None admin] RBAC: Authorizing `identity:get_user(user_id=de77771d93024bfdb51885538564c97a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:52.851692 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead3ddcb-7d1b-4b57-a0eb-97eec0dbe7fd None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:52.854095 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 98/202] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Oct 27 09:23:52 2022] GET /identity/v3/users/de77771d93024bfdb51885538564c97a => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:23:52.859877 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6720ce1d-7bcc-4a7f-8fd0-a1473ec2c1d4 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:52.979729 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6720ce1d-7bcc-4a7f-8fd0-a1473ec2c1d4 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:53.019417 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6991c42-2f51-49d4-8d09-89255f28ec60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:53.020034 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6991c42-2f51-49d4-8d09-89255f28ec60 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:53.020317 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6991c42-2f51-49d4-8d09-89255f28ec60 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:53.020613 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6991c42-2f51-49d4-8d09-89255f28ec60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:53.078163 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:53.094839 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6991c42-2f51-49d4-8d09-89255f28ec60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:53.096697 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6991c42-2f51-49d4-8d09-89255f28ec60 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:53.205808 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 105/203] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Oct 27 09:23:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 27 09:23:53.211159 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead3ddcb-7d1b-4b57-a0eb-97eec0dbe7fd None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:53.315532 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead3ddcb-7d1b-4b57-a0eb-97eec0dbe7fd None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:53.353198 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f73a612-326e-44ab-8ebd-ac84c71df3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:53.354550 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73a612-326e-44ab-8ebd-ac84c71df3d3 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:53.355036 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73a612-326e-44ab-8ebd-ac84c71df3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:53.355530 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73a612-326e-44ab-8ebd-ac84c71df3d3 None admin] PATH_INFO: `/v3/projects/e6e4e23ca6ee4fc0a033668401d7ba30` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:53.356880 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f73a612-326e-44ab-8ebd-ac84c71df3d3 None admin] RBAC: Authorizing `identity:get_project(project_id=e6e4e23ca6ee4fc0a033668401d7ba30)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:53.358491 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f73a612-326e-44ab-8ebd-ac84c71df3d3 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:53.360608 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 99/204] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Oct 27 09:23:53 2022] GET /identity/v3/projects/e6e4e23ca6ee4fc0a033668401d7ba30 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:53.366604 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6991c42-2f51-49d4-8d09-89255f28ec60 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:53.454531 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:53.471573 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6991c42-2f51-49d4-8d09-89255f28ec60 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:53.506397 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfc8586-bed5-4ff8-96a4-b06cfe9d087b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:53.507008 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc8586-bed5-4ff8-96a4-b06cfe9d087b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:53.507284 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc8586-bed5-4ff8-96a4-b06cfe9d087b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:53.507556 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc8586-bed5-4ff8-96a4-b06cfe9d087b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:53.508272 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfc8586-bed5-4ff8-96a4-b06cfe9d087b None admin] RBAC: Adding query filter params (role.id=eea7cd23e01b4506a9eb6ab86f9adcd2, scope.project.id=e6e4e23ca6ee4fc0a033668401d7ba30, user.id=de77771d93024bfdb51885538564c97a) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:23:53.508605 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfc8586-bed5-4ff8-96a4-b06cfe9d087b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:53.510138 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfc8586-bed5-4ff8-96a4-b06cfe9d087b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:53.516290 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 106/205] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Oct 27 09:23:53 2022] GET /identity/v3/role_assignments?user.id=de77771d93024bfdb51885538564c97a&scope.project.id=e6e4e23ca6ee4fc0a033668401d7ba30&role.id=eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:54.296744 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66bbbd0-ff81-46b8-9ccc-acd45b208ee9 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:54.297028 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66bbbd0-ff81-46b8-9ccc-acd45b208ee9 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:54.297299 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66bbbd0-ff81-46b8-9ccc-acd45b208ee9 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:54.298786 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 100/206] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:23:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:23:54.303769 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9ddd6-4c22-4d65-9799-a4b9351f7bfb None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:54.304062 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9ddd6-4c22-4d65-9799-a4b9351f7bfb None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:54.304347 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9ddd6-4c22-4d65-9799-a4b9351f7bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:54.325447 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-dcc9ddd6-4c22-4d65-9799-a4b9351f7bfb None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:23:54.326683 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-dcc9ddd6-4c22-4d65-9799-a4b9351f7bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:23:54.398689 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 107/207] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:23:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 27 09:23:54.450946 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce12afb9-759b-4390-b841-8dc627567dfe None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:54.451453 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce12afb9-759b-4390-b841-8dc627567dfe None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:54.451453 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce12afb9-759b-4390-b841-8dc627567dfe None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:54.453016 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 101/208] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:23:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:23:54.458156 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76182f0-df06-493b-9394-618eee31eb73 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:54.458444 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76182f0-df06-493b-9394-618eee31eb73 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:54.458716 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76182f0-df06-493b-9394-618eee31eb73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:54.476868 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-e76182f0-df06-493b-9394-618eee31eb73 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:23:54.478522 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e76182f0-df06-493b-9394-618eee31eb73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:23:54.543946 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 108/209] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:23:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:23:54.673000 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce12afb9-759b-4390-b841-8dc627567dfe None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:54.732411 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:54.775403 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce12afb9-759b-4390-b841-8dc627567dfe None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:54.808805 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76182f0-df06-493b-9394-618eee31eb73 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:54.810168 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0992465-a927-4eb8-87c3-d104017dedec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:54.810821 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0992465-a927-4eb8-87c3-d104017dedec None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:54.811083 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0992465-a927-4eb8-87c3-d104017dedec None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:54.811352 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0992465-a927-4eb8-87c3-d104017dedec None admin] PATH_INFO: `/v3/roles/member` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:54.820633 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:54.822546 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0992465-a927-4eb8-87c3-d104017dedec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:23:54.827512 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-d0992465-a927-4eb8-87c3-d104017dedec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 27 09:23:54.827512 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:23:54.827512 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:23:54.827512 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:23:54.827512 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:23:54.827512 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:23:54.827512 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:23:54.827512 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:23:54.827512 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:23:54.827512 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:23:54.827512 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:23:54.827512 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:23:54.827512 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:23:54.827512 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:23:54.827512 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:23:54.827512 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:23:54.828587 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:23:54.828587 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:23:54.828587 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:23:54.828587 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:23:54.828587 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:23:54.828587 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:23:54.828587 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:23:54.828587 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:23:54.828587 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:23:54.828587 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:23:54.828587 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:23:54.828587 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:23:54.828587 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:23:54.828587 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:23:54.828587 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:23:54.828587 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:23:54.829635 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:23:54.829635 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:23:54.829635 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:23:54.829635 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:23:54.829635 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:23:54.829635 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:23:54.829635 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:23:54.829635 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:23:54.829635 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:23:54.829635 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:23:54.829635 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 27 09:23:54.829635 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:23:54.829635 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 102/210] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Oct 27 09:23:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:23:54.836167 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0992465-a927-4eb8-87c3-d104017dedec None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:54.918080 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76182f0-df06-493b-9394-618eee31eb73 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:54.945652 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0992465-a927-4eb8-87c3-d104017dedec None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:54.956211 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce130edf-f592-4f55-8af6-2bf0394dd9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:54.956882 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce130edf-f592-4f55-8af6-2bf0394dd9e0 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:54.957161 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce130edf-f592-4f55-8af6-2bf0394dd9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:54.957431 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce130edf-f592-4f55-8af6-2bf0394dd9e0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:54.958539 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce130edf-f592-4f55-8af6-2bf0394dd9e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:54.959967 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce130edf-f592-4f55-8af6-2bf0394dd9e0 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:54.961763 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 109/211] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:23:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:23:54.966895 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:54.967367 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce130edf-f592-4f55-8af6-2bf0394dd9e0 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:54.985814 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c745b124-4bfe-4b77-b462-d456cad121d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:54.986420 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c745b124-4bfe-4b77-b462-d456cad121d0 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:54.986711 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c745b124-4bfe-4b77-b462-d456cad121d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:54.986961 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c745b124-4bfe-4b77-b462-d456cad121d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:54.987617 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c745b124-4bfe-4b77-b462-d456cad121d0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:23:54.987983 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c745b124-4bfe-4b77-b462-d456cad121d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:54.989326 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c745b124-4bfe-4b77-b462-d456cad121d0 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:54.997776 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 103/212] 10.222.0.70 () {56 vars in 1101 bytes} [Thu Oct 27 09:23:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:23:55.003151 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c745b124-4bfe-4b77-b462-d456cad121d0 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:55.049580 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:55.071021 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce130edf-f592-4f55-8af6-2bf0394dd9e0 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:55.101240 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c745b124-4bfe-4b77-b462-d456cad121d0 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:55.104800 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dc4366-c8e1-4df9-9702-7e047355d08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:55.105710 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc4366-c8e1-4df9-9702-7e047355d08b None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:55.105992 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc4366-c8e1-4df9-9702-7e047355d08b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:55.106261 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc4366-c8e1-4df9-9702-7e047355d08b None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:55.106907 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dc4366-c8e1-4df9-9702-7e047355d08b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:55.108333 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dc4366-c8e1-4df9-9702-7e047355d08b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:55.127943 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 110/213] 10.222.0.70 () {60 vars in 1132 bytes} [Thu Oct 27 09:23:54 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:23:55.129652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4126e-1e53-4629-bccb-8933e6b71571 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:55.130183 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4126e-1e53-4629-bccb-8933e6b71571 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:55.130183 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4126e-1e53-4629-bccb-8933e6b71571 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:55.131444 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 111/214] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:23:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:23:55.137418 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c43dba-138b-46cc-81c5-b755d41d56d8 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:55.137726 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c43dba-138b-46cc-81c5-b755d41d56d8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:55.137967 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c43dba-138b-46cc-81c5-b755d41d56d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:55.140281 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f00db1-d302-461d-9c12-0a49f261c7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:55.140966 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f00db1-d302-461d-9c12-0a49f261c7f4 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:55.141239 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f00db1-d302-461d-9c12-0a49f261c7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:55.141530 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f00db1-d302-461d-9c12-0a49f261c7f4 None admin] PATH_INFO: `/v3/users/5625732a380241698f525db7052bab5a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:55.160473 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-c9c43dba-138b-46cc-81c5-b755d41d56d8 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:23:55.161372 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f00db1-d302-461d-9c12-0a49f261c7f4 None admin] RBAC: Authorizing `identity:get_user(user_id=5625732a380241698f525db7052bab5a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:55.161709 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c9c43dba-138b-46cc-81c5-b755d41d56d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:23:55.162884 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f00db1-d302-461d-9c12-0a49f261c7f4 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:55.170295 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 104/215] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Oct 27 09:23:55 2022] GET /identity/v3/users/5625732a380241698f525db7052bab5a => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:55.171863 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f00db1-d302-461d-9c12-0a49f261c7f4 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:55.220467 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:55.239375 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 112/216] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:23:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 27 09:23:55.294921 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f00db1-d302-461d-9c12-0a49f261c7f4 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:55.332323 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f63fd-b4a3-45f8-94e1-b3e5d3937097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:55.332952 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f63fd-b4a3-45f8-94e1-b3e5d3937097 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:55.333256 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f63fd-b4a3-45f8-94e1-b3e5d3937097 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:55.333522 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f63fd-b4a3-45f8-94e1-b3e5d3937097 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:55.334239 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3f63fd-b4a3-45f8-94e1-b3e5d3937097 None admin] RBAC: Adding query filter params (role.id=eea7cd23e01b4506a9eb6ab86f9adcd2, user.id=5625732a380241698f525db7052bab5a, scope.system=all) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:23:55.334540 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3f63fd-b4a3-45f8-94e1-b3e5d3937097 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:55.336044 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3f63fd-b4a3-45f8-94e1-b3e5d3937097 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:55.341999 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 105/217] 10.222.0.70 () {56 vars in 1308 bytes} [Thu Oct 27 09:23:55 2022] GET /identity/v3/role_assignments?user.id=5625732a380241698f525db7052bab5a&scope.system=all&role.id=eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:55.507502 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c43dba-138b-46cc-81c5-b755d41d56d8 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:55.573202 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:55.614274 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c43dba-138b-46cc-81c5-b755d41d56d8 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:55.648922 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaeda89-180a-4ebb-b7c8-52e12eb79132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:55.650129 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaeda89-180a-4ebb-b7c8-52e12eb79132 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:55.650400 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaeda89-180a-4ebb-b7c8-52e12eb79132 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:55.650664 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaeda89-180a-4ebb-b7c8-52e12eb79132 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:55.663268 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaeda89-180a-4ebb-b7c8-52e12eb79132 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:23:55.668872 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-bfaeda89-180a-4ebb-b7c8-52e12eb79132 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 27 09:23:55.668872 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:23:55.668872 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:23:55.668872 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:23:55.668872 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:23:55.668872 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:23:55.668872 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:23:55.668872 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:23:55.668872 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:23:55.668872 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:23:55.668872 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:23:55.668872 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:23:55.668872 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:23:55.668872 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:23:55.668872 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:23:55.668872 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:23:55.669731 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:23:55.669731 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:23:55.669731 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:23:55.669731 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:23:55.669731 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:23:55.669731 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:23:55.669731 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:23:55.669731 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:23:55.669731 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:23:55.669731 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:23:55.669731 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:23:55.669731 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:23:55.669731 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:23:55.669731 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:23:55.669731 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:23:55.669731 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:23:55.670516 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:23:55.670516 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:23:55.670516 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:23:55.670516 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:23:55.670516 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:23:55.670516 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:23:55.670516 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:23:55.670516 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:23:55.670516 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:23:55.670516 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:23:55.670516 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 27 09:23:55.670516 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:23:55.670516 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 113/218] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Oct 27 09:23:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:23:55.675929 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f63fd-b4a3-45f8-94e1-b3e5d3937097 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:55.779937 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f63fd-b4a3-45f8-94e1-b3e5d3937097 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:55.781944 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:55.819534 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5072618-c8b7-49ec-9686-c1ccf6b696b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:55.820141 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5072618-c8b7-49ec-9686-c1ccf6b696b4 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:55.820416 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5072618-c8b7-49ec-9686-c1ccf6b696b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:55.820712 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5072618-c8b7-49ec-9686-c1ccf6b696b4 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:55.821394 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5072618-c8b7-49ec-9686-c1ccf6b696b4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:23:55.821692 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5072618-c8b7-49ec-9686-c1ccf6b696b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:55.823242 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5072618-c8b7-49ec-9686-c1ccf6b696b4 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:55.831850 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 106/219] 10.222.0.70 () {56 vars in 1101 bytes} [Thu Oct 27 09:23:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:55.838132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaeda89-180a-4ebb-b7c8-52e12eb79132 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:55.948254 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaeda89-180a-4ebb-b7c8-52e12eb79132 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:55.973388 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:55.984613 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54faf647-4e1b-42e8-b59d-2dc5137759aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:55.985321 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54faf647-4e1b-42e8-b59d-2dc5137759aa None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:55.985601 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54faf647-4e1b-42e8-b59d-2dc5137759aa None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:55.985876 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54faf647-4e1b-42e8-b59d-2dc5137759aa None admin] PATH_INFO: `/v3/users/41179171b7644ed8a51bc41ab1678469` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:55.987045 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54faf647-4e1b-42e8-b59d-2dc5137759aa None admin] RBAC: Authorizing `identity:get_user(user_id=41179171b7644ed8a51bc41ab1678469)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:55.988813 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54faf647-4e1b-42e8-b59d-2dc5137759aa None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:55.995474 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 114/220] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Oct 27 09:23:55 2022] GET /identity/v3/users/41179171b7644ed8a51bc41ab1678469 => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:55.996718 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5072618-c8b7-49ec-9686-c1ccf6b696b4 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:56.102706 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5072618-c8b7-49ec-9686-c1ccf6b696b4 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:56.141497 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59ba4a-1ce2-454c-885b-c6fc6400eaa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:56.142106 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59ba4a-1ce2-454c-885b-c6fc6400eaa4 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:56.142384 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59ba4a-1ce2-454c-885b-c6fc6400eaa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:56.142652 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59ba4a-1ce2-454c-885b-c6fc6400eaa4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:56.174647 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:56.217323 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc59ba4a-1ce2-454c-885b-c6fc6400eaa4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:56.218718 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc59ba4a-1ce2-454c-885b-c6fc6400eaa4 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:56.327879 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 107/221] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Oct 27 09:23:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 27 09:23:56.328829 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54faf647-4e1b-42e8-b59d-2dc5137759aa None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:56.432193 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54faf647-4e1b-42e8-b59d-2dc5137759aa None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:56.466765 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee8f56e-81b0-4e8d-96f6-f35d2ce92510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:56.467447 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee8f56e-81b0-4e8d-96f6-f35d2ce92510 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:56.467670 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee8f56e-81b0-4e8d-96f6-f35d2ce92510 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:56.467938 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee8f56e-81b0-4e8d-96f6-f35d2ce92510 None admin] PATH_INFO: `/v3/projects/e6e4e23ca6ee4fc0a033668401d7ba30` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:56.469082 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee8f56e-81b0-4e8d-96f6-f35d2ce92510 None admin] RBAC: Authorizing `identity:get_project(project_id=e6e4e23ca6ee4fc0a033668401d7ba30)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:56.470770 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee8f56e-81b0-4e8d-96f6-f35d2ce92510 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:56.472384 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 115/222] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Oct 27 09:23:56 2022] GET /identity/v3/projects/e6e4e23ca6ee4fc0a033668401d7ba30 => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:56.478399 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59ba4a-1ce2-454c-885b-c6fc6400eaa4 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:56.541231 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:56.583718 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59ba4a-1ce2-454c-885b-c6fc6400eaa4 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:56.617757 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac76cd18-e1bb-4e07-a56c-7f94497b7cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:56.618905 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac76cd18-e1bb-4e07-a56c-7f94497b7cdc None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:56.619182 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac76cd18-e1bb-4e07-a56c-7f94497b7cdc None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:56.619447 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac76cd18-e1bb-4e07-a56c-7f94497b7cdc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:56.620161 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac76cd18-e1bb-4e07-a56c-7f94497b7cdc None admin] RBAC: Adding query filter params (role.id=f1938eac464c4f22bb492ea2fdf4899a, scope.project.id=e6e4e23ca6ee4fc0a033668401d7ba30, user.id=41179171b7644ed8a51bc41ab1678469) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:23:56.620462 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac76cd18-e1bb-4e07-a56c-7f94497b7cdc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:56.621940 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac76cd18-e1bb-4e07-a56c-7f94497b7cdc None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:56.627997 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 108/223] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Oct 27 09:23:56 2022] GET /identity/v3/role_assignments?user.id=41179171b7644ed8a51bc41ab1678469&scope.project.id=e6e4e23ca6ee4fc0a033668401d7ba30&role.id=f1938eac464c4f22bb492ea2fdf4899a => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:57.534044 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4823fb67-dceb-417a-bb36-595a04b2b624 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:57.534536 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4823fb67-dceb-417a-bb36-595a04b2b624 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:57.534751 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4823fb67-dceb-417a-bb36-595a04b2b624 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:57.536232 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 116/224] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:23:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:23:57.536337 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82170811-5253-4574-8914-8bd6a62f27ad None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:57.536733 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82170811-5253-4574-8914-8bd6a62f27ad None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:57.536826 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82170811-5253-4574-8914-8bd6a62f27ad None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:57.538195 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 109/225] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:23:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:23:57.541434 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c8171-bc88-4e11-bd00-8830db5f64ef None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:57.541716 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c8171-bc88-4e11-bd00-8830db5f64ef None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:57.541986 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c8171-bc88-4e11-bd00-8830db5f64ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:57.543990 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e3a6a7-f9f9-43e3-ba05-a4834f255cb4 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:57.544274 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e3a6a7-f9f9-43e3-ba05-a4834f255cb4 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:57.544567 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e3a6a7-f9f9-43e3-ba05-a4834f255cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:57.560387 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f74c8171-bc88-4e11-bd00-8830db5f64ef None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:23:57.561584 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f74c8171-bc88-4e11-bd00-8830db5f64ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:23:57.563268 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-d3e3a6a7-f9f9-43e3-ba05-a4834f255cb4 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:23:57.564670 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d3e3a6a7-f9f9-43e3-ba05-a4834f255cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:23:57.620443 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:57.632679 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 117/226] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:23:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:23:57.634071 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e9e031-9e47-4ac1-9d8e-03a802c5b600 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:57.634345 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e9e031-9e47-4ac1-9d8e-03a802c5b600 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:57.634615 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e9e031-9e47-4ac1-9d8e-03a802c5b600 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:57.636038 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 118/227] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:23:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:23:57.639433 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 110/228] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:23:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 27 09:23:57.641002 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53563bc-aa1a-4e42-ad2a-259773fc75e6 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:57.641288 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53563bc-aa1a-4e42-ad2a-259773fc75e6 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:57.641564 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53563bc-aa1a-4e42-ad2a-259773fc75e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:57.659172 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-b53563bc-aa1a-4e42-ad2a-259773fc75e6 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:23:57.660349 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b53563bc-aa1a-4e42-ad2a-259773fc75e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:23:57.725489 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 119/229] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:23:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:23:57.897293 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e3a6a7-f9f9-43e3-ba05-a4834f255cb4 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:57.906990 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53563bc-aa1a-4e42-ad2a-259773fc75e6 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:57.958348 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:58.004707 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e3a6a7-f9f9-43e3-ba05-a4834f255cb4 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:58.014315 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53563bc-aa1a-4e42-ad2a-259773fc75e6 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:58.034206 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:58.039651 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243bbe65-9ad4-436c-9fe8-074acbf1297a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:58.040992 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243bbe65-9ad4-436c-9fe8-074acbf1297a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:58.041269 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243bbe65-9ad4-436c-9fe8-074acbf1297a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:58.041531 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243bbe65-9ad4-436c-9fe8-074acbf1297a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:58.048693 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0da8da-7133-4100-b957-af29216b68aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:58.049367 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0da8da-7133-4100-b957-af29216b68aa None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:58.049647 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0da8da-7133-4100-b957-af29216b68aa None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:58.049915 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0da8da-7133-4100-b957-af29216b68aa None admin] PATH_INFO: `/v3/domains/default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:58.051001 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0da8da-7133-4100-b957-af29216b68aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:58.051660 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243bbe65-9ad4-436c-9fe8-074acbf1297a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:23:58.052427 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0da8da-7133-4100-b957-af29216b68aa None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:58.054024 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 120/230] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:23:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:23:58.054775 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0da8da-7133-4100-b957-af29216b68aa None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:58.056731 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-243bbe65-9ad4-436c-9fe8-074acbf1297a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 27 09:23:58.056731 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:23:58.056731 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:23:58.056731 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:23:58.056731 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:23:58.056731 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:23:58.056731 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:23:58.056731 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:23:58.056731 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:23:58.056731 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:23:58.056731 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:23:58.056731 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:23:58.056731 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:23:58.056731 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:23:58.056731 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:23:58.056731 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:23:58.057575 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:23:58.057575 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:23:58.057575 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:23:58.057575 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:23:58.057575 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:23:58.057575 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:23:58.057575 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:23:58.057575 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:23:58.057575 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:23:58.057575 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:23:58.057575 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:23:58.057575 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:23:58.057575 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:23:58.057575 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:23:58.057575 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:23:58.057575 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:23:58.058335 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:23:58.058335 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:23:58.058335 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:23:58.058335 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:23:58.058335 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:23:58.058335 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:23:58.058335 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:23:58.058335 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:23:58.058335 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:23:58.058335 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:23:58.058335 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 27 09:23:58.058335 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:23:58.058335 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 111/231] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Oct 27 09:23:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:23:58.059627 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243bbe65-9ad4-436c-9fe8-074acbf1297a None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:58.167192 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0da8da-7133-4100-b957-af29216b68aa None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:58.169182 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243bbe65-9ad4-436c-9fe8-074acbf1297a None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:58.189155 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:58.203103 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a5fe85-1886-42d1-b8d7-8c6583c330ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:58.203737 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a5fe85-1886-42d1-b8d7-8c6583c330ad None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:58.204013 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a5fe85-1886-42d1-b8d7-8c6583c330ad None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:58.204280 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a5fe85-1886-42d1-b8d7-8c6583c330ad None admin] PATH_INFO: `/v3/roles/member` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:58.208127 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09192e2f-c41a-4acb-bb51-7f7f203e995f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:58.208868 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09192e2f-c41a-4acb-bb51-7f7f203e995f None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:58.209245 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09192e2f-c41a-4acb-bb51-7f7f203e995f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:58.209588 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09192e2f-c41a-4acb-bb51-7f7f203e995f None admin] PATH_INFO: `/v3/groups` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:58.210270 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09192e2f-c41a-4acb-bb51-7f7f203e995f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:58.212407 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09192e2f-c41a-4acb-bb51-7f7f203e995f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:58.215365 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a5fe85-1886-42d1-b8d7-8c6583c330ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:23:58.221042 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-b8a5fe85-1886-42d1-b8d7-8c6583c330ad None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 27 09:23:58.221042 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:23:58.221042 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:23:58.221042 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:23:58.221042 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:23:58.221042 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:23:58.221042 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:23:58.221042 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:23:58.221042 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:23:58.221042 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:23:58.221042 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:23:58.221042 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:23:58.221042 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:23:58.221042 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:23:58.221042 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:23:58.221042 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:23:58.222019 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:23:58.222019 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:23:58.222019 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:23:58.222019 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:23:58.222019 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:23:58.222019 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:23:58.222019 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:23:58.222019 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:23:58.222019 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:23:58.222019 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:23:58.222019 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:23:58.222019 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:23:58.222019 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:23:58.222019 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:23:58.222019 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:23:58.222019 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:23:58.225520 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:23:58.225520 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:23:58.225520 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:23:58.225520 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:23:58.225520 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:23:58.225520 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:23:58.225520 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:23:58.225520 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:23:58.225520 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:23:58.225520 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:23:58.225520 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 27 09:23:58.225520 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:23:58.225520 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 121/232] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Oct 27 09:23:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:23:58.225520 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a5fe85-1886-42d1-b8d7-8c6583c330ad None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:58.235783 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 112/233] 10.222.0.70 () {60 vars in 1134 bytes} [Thu Oct 27 09:23:58 2022] POST /identity/v3/groups => generated 229 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:23:58.245642 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09192e2f-c41a-4acb-bb51-7f7f203e995f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:58.298074 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:58.362106 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09192e2f-c41a-4acb-bb51-7f7f203e995f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:58.365492 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a5fe85-1886-42d1-b8d7-8c6583c330ad None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:58.402399 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832b3342-98f4-493f-b710-38540ef8e8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:58.403015 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b3342-98f4-493f-b710-38540ef8e8cd None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:58.403291 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b3342-98f4-493f-b710-38540ef8e8cd None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:58.403558 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b3342-98f4-493f-b710-38540ef8e8cd None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:58.404218 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832b3342-98f4-493f-b710-38540ef8e8cd None admin] RBAC: Adding query filter params (name=member) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:23:58.404535 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832b3342-98f4-493f-b710-38540ef8e8cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:58.406005 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832b3342-98f4-493f-b710-38540ef8e8cd None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:58.406567 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96762a32-5fe4-4e9a-915c-d173c08f0c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:58.407553 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96762a32-5fe4-4e9a-915c-d173c08f0c1a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:58.407905 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96762a32-5fe4-4e9a-915c-d173c08f0c1a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:58.408252 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96762a32-5fe4-4e9a-915c-d173c08f0c1a None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:58.409112 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96762a32-5fe4-4e9a-915c-d173c08f0c1a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:23:58.409485 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96762a32-5fe4-4e9a-915c-d173c08f0c1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:58.410958 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96762a32-5fe4-4e9a-915c-d173c08f0c1a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:58.416281 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 113/234] 10.222.0.70 () {56 vars in 1101 bytes} [Thu Oct 27 09:23:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:58.422174 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832b3342-98f4-493f-b710-38540ef8e8cd None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:58.425367 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 122/235] 10.222.0.70 () {56 vars in 1101 bytes} [Thu Oct 27 09:23:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:58.432267 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96762a32-5fe4-4e9a-915c-d173c08f0c1a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:58.481728 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:58.544124 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832b3342-98f4-493f-b710-38540ef8e8cd None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:58.549743 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96762a32-5fe4-4e9a-915c-d173c08f0c1a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:58.562544 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:58.580449 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30ea25d-3590-4840-8d68-5d9a19a6af26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:58.581233 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ea25d-3590-4840-8d68-5d9a19a6af26 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:58.581403 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ea25d-3590-4840-8d68-5d9a19a6af26 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:58.581677 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ea25d-3590-4840-8d68-5d9a19a6af26 None admin] PATH_INFO: `/v3/users/5625732a380241698f525db7052bab5a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:58.582883 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30ea25d-3590-4840-8d68-5d9a19a6af26 None admin] RBAC: Authorizing `identity:get_user(user_id=5625732a380241698f525db7052bab5a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:58.584351 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30ea25d-3590-4840-8d68-5d9a19a6af26 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:58.591023 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 114/236] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Oct 27 09:23:58 2022] GET /identity/v3/users/5625732a380241698f525db7052bab5a => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:58.591150 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c61cbef-b1d3-4bac-820a-37c1bb7d620a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:58.591150 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c61cbef-b1d3-4bac-820a-37c1bb7d620a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:58.591150 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c61cbef-b1d3-4bac-820a-37c1bb7d620a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:58.591150 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c61cbef-b1d3-4bac-820a-37c1bb7d620a None admin] PATH_INFO: `/v3/users/567a0a16618844dda14ddbd64da73c4b` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:58.592883 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30ea25d-3590-4840-8d68-5d9a19a6af26 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:58.603060 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c61cbef-b1d3-4bac-820a-37c1bb7d620a None admin] RBAC: Authorizing `identity:get_user(user_id=567a0a16618844dda14ddbd64da73c4b)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:58.604339 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c61cbef-b1d3-4bac-820a-37c1bb7d620a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:58.606197 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 123/237] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Oct 27 09:23:58 2022] GET /identity/v3/users/567a0a16618844dda14ddbd64da73c4b => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:23:58.611582 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c61cbef-b1d3-4bac-820a-37c1bb7d620a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:23:58.704100 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30ea25d-3590-4840-8d68-5d9a19a6af26 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:58.712945 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:23:58.717524 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c61cbef-b1d3-4bac-820a-37c1bb7d620a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:23:58.739044 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe5d0db-7f00-4d9b-af39-e8f9383ba12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:58.739705 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5d0db-7f00-4d9b-af39-e8f9383ba12f None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:58.739985 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5d0db-7f00-4d9b-af39-e8f9383ba12f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:58.740260 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5d0db-7f00-4d9b-af39-e8f9383ba12f None admin] PATH_INFO: `/v3/system/users/5625732a380241698f525db7052bab5a/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:58.741697 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe5d0db-7f00-4d9b-af39-e8f9383ba12f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5625732a380241698f525db7052bab5a, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:58.743631 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe5d0db-7f00-4d9b-af39-e8f9383ba12f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:58.753825 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 115/238] 10.222.0.70 () {58 vars in 1334 bytes} [Thu Oct 27 09:23:58 2022] PUT /identity/v3/system/users/5625732a380241698f525db7052bab5a/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:23:58.764144 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd5a2b2-91d9-41f9-b6a0-240b06b095d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:23:58.764144 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd5a2b2-91d9-41f9-b6a0-240b06b095d1 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:23:58.764144 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd5a2b2-91d9-41f9-b6a0-240b06b095d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:23:58.764144 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd5a2b2-91d9-41f9-b6a0-240b06b095d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:23:58.764468 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd5a2b2-91d9-41f9-b6a0-240b06b095d1 None admin] RBAC: Adding query filter params (role.id=f1938eac464c4f22bb492ea2fdf4899a, user.id=567a0a16618844dda14ddbd64da73c4b, scope.system=all) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:23:58.764468 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd5a2b2-91d9-41f9-b6a0-240b06b095d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:23:58.764468 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd5a2b2-91d9-41f9-b6a0-240b06b095d1 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:23:58.772726 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 124/239] 10.222.0.70 () {56 vars in 1308 bytes} [Thu Oct 27 09:23:58 2022] GET /identity/v3/role_assignments?user.id=567a0a16618844dda14ddbd64da73c4b&scope.system=all&role.id=f1938eac464c4f22bb492ea2fdf4899a => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:24:00.448052 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37a19b-99bb-4c1b-96d3-f0a53ab84c28 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:00.448773 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37a19b-99bb-4c1b-96d3-f0a53ab84c28 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:00.448773 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37a19b-99bb-4c1b-96d3-f0a53ab84c28 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:00.450119 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 116/240] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:00.455310 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53ef0d-4b69-44a6-8008-8170d59c20b9 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:00.455724 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53ef0d-4b69-44a6-8008-8170d59c20b9 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:00.455973 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53ef0d-4b69-44a6-8008-8170d59c20b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:00.474483 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-ce53ef0d-4b69-44a6-8008-8170d59c20b9 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:00.475666 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ce53ef0d-4b69-44a6-8008-8170d59c20b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:00.503630 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:00.547642 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 125/241] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 27 09:24:00.819038 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e37a19b-99bb-4c1b-96d3-f0a53ab84c28 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:00.913239 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b05cb7-6156-4589-a515-ee90c40ca934 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:00.913614 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b05cb7-6156-4589-a515-ee90c40ca934 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:00.913960 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b05cb7-6156-4589-a515-ee90c40ca934 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:00.916157 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 126/242] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 27 09:24:00.921473 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4841a1a4-6cf6-47cd-8c94-ecc5d830862c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:00.921845 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4841a1a4-6cf6-47cd-8c94-ecc5d830862c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:00.922219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4841a1a4-6cf6-47cd-8c94-ecc5d830862c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:00.931446 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e37a19b-99bb-4c1b-96d3-f0a53ab84c28 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:00.940030 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-4841a1a4-6cf6-47cd-8c94-ecc5d830862c None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:00.941231 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4841a1a4-6cf6-47cd-8c94-ecc5d830862c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:00.965217 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa5732c-71ce-4bed-952c-18fefd7608bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:00.965833 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa5732c-71ce-4bed-952c-18fefd7608bc None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:00.966104 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa5732c-71ce-4bed-952c-18fefd7608bc None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:00.966366 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa5732c-71ce-4bed-952c-18fefd7608bc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:00.967431 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa5732c-71ce-4bed-952c-18fefd7608bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:00.968893 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa5732c-71ce-4bed-952c-18fefd7608bc None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:00.975161 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 117/243] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:24:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:00.975161 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859229f9-10a4-4c62-9626-decf4e5104d6 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:00.975161 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859229f9-10a4-4c62-9626-decf4e5104d6 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:00.975161 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859229f9-10a4-4c62-9626-decf4e5104d6 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:00.975161 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 118/244] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:00.978070 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859229f9-10a4-4c62-9626-decf4e5104d6 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:01.010247 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 127/245] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 27 09:24:01.011477 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56646dcc-d481-4145-9b52-f114909601db None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:01.011766 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56646dcc-d481-4145-9b52-f114909601db None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:01.012038 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56646dcc-d481-4145-9b52-f114909601db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:01.022575 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:01.030050 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-56646dcc-d481-4145-9b52-f114909601db None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:01.031318 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-56646dcc-d481-4145-9b52-f114909601db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:01.088533 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859229f9-10a4-4c62-9626-decf4e5104d6 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:01.108425 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 128/246] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:24:01.130301 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f4b5ae-cca3-4280-88d4-07204f3ef9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:01.130959 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4b5ae-cca3-4280-88d4-07204f3ef9b0 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:01.131247 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4b5ae-cca3-4280-88d4-07204f3ef9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:01.131518 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4b5ae-cca3-4280-88d4-07204f3ef9b0 None admin] PATH_INFO: `/v3/groups` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:01.132168 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f4b5ae-cca3-4280-88d4-07204f3ef9b0 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:01.133669 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f4b5ae-cca3-4280-88d4-07204f3ef9b0 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:01.151831 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 119/247] 10.222.0.70 () {60 vars in 1134 bytes} [Thu Oct 27 09:24:00 2022] POST /identity/v3/groups => generated 226 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:24:01.283734 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56646dcc-d481-4145-9b52-f114909601db None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:01.310772 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:01.379772 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f4b5ae-cca3-4280-88d4-07204f3ef9b0 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:01.398996 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56646dcc-d481-4145-9b52-f114909601db None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:01.439323 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27eff70-e913-4f7b-9833-770ef029bd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:01.439955 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27eff70-e913-4f7b-9833-770ef029bd55 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:01.440229 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27eff70-e913-4f7b-9833-770ef029bd55 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:01.440508 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27eff70-e913-4f7b-9833-770ef029bd55 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:01.452122 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27eff70-e913-4f7b-9833-770ef029bd55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:24:01.457511 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-e27eff70-e913-4f7b-9833-770ef029bd55 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 27 09:24:01.457511 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:01.457511 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:01.457511 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:01.457511 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:01.457511 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:01.457511 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:01.457511 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:01.457511 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:01.457511 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:01.457511 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:01.457511 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:01.457511 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:24:01.457511 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:24:01.457511 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:24:01.457511 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:24:01.458805 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:24:01.458805 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:24:01.458805 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:24:01.458805 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:24:01.458805 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:01.458805 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:01.458805 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:01.458805 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:01.458805 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:01.458805 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:01.458805 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:01.458805 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:01.458805 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:01.458805 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:01.458805 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:01.458805 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:01.459583 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:01.459583 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:01.459583 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:01.459583 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:01.459583 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:24:01.459583 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:24:01.459583 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:24:01.459583 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:24:01.459583 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:24:01.459583 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:24:01.459583 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 27 09:24:01.459583 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:01.459583 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 129/248] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Oct 27 09:24:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:01.467217 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27eff70-e913-4f7b-9833-770ef029bd55 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:01.512479 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:01.518014 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f4b5ae-cca3-4280-88d4-07204f3ef9b0 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:01.562752 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abad534d-d730-4a8f-b2a6-6b57e64937ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:01.563913 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad534d-d730-4a8f-b2a6-6b57e64937ce None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:01.564198 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad534d-d730-4a8f-b2a6-6b57e64937ce None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:01.564471 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad534d-d730-4a8f-b2a6-6b57e64937ce None admin] PATH_INFO: `/v3/roles/member` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:01.581554 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abad534d-d730-4a8f-b2a6-6b57e64937ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:24:01.584664 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:01.589662 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-abad534d-d730-4a8f-b2a6-6b57e64937ce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 27 09:24:01.589662 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:01.589662 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:01.589662 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:01.589662 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:01.589662 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:01.589662 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:01.589662 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:01.589662 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:01.589662 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:01.589662 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:01.589662 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:01.589662 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:24:01.589662 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:24:01.589662 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:24:01.589662 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:24:01.590533 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:24:01.590533 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:24:01.590533 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:24:01.590533 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:24:01.590533 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:01.590533 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:01.590533 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:01.590533 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:01.590533 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:01.590533 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:01.590533 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:01.590533 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:01.590533 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:01.590533 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:01.590533 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:01.590533 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:01.591582 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:01.591582 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:01.591582 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:01.591582 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:01.591582 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:24:01.591582 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:24:01.591582 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:24:01.591582 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:24:01.591582 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:24:01.591582 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:24:01.591582 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 27 09:24:01.591582 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:01.594509 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 120/249] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Oct 27 09:24:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 09:24:01.601161 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abad534d-d730-4a8f-b2a6-6b57e64937ce None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:01.609959 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27eff70-e913-4f7b-9833-770ef029bd55 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:01.649973 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d000b2-f53d-4008-9b71-4267a04b8446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:01.650499 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d000b2-f53d-4008-9b71-4267a04b8446 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:01.650773 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d000b2-f53d-4008-9b71-4267a04b8446 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:01.651041 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d000b2-f53d-4008-9b71-4267a04b8446 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:01.651704 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d000b2-f53d-4008-9b71-4267a04b8446 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:01.652004 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d000b2-f53d-4008-9b71-4267a04b8446 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:01.654012 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d000b2-f53d-4008-9b71-4267a04b8446 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:01.662216 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 130/250] 10.222.0.70 () {56 vars in 1101 bytes} [Thu Oct 27 09:24:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:24:01.669174 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d000b2-f53d-4008-9b71-4267a04b8446 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:01.738318 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abad534d-d730-4a8f-b2a6-6b57e64937ce None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:01.791043 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd508d02-ad15-4614-ba7f-17767d5bdc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:01.791657 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd508d02-ad15-4614-ba7f-17767d5bdc8c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:01.791944 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd508d02-ad15-4614-ba7f-17767d5bdc8c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:01.792233 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd508d02-ad15-4614-ba7f-17767d5bdc8c None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:01.792894 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd508d02-ad15-4614-ba7f-17767d5bdc8c None admin] RBAC: Adding query filter params (name=member) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:01.793196 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd508d02-ad15-4614-ba7f-17767d5bdc8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:01.794727 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd508d02-ad15-4614-ba7f-17767d5bdc8c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:01.804970 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d000b2-f53d-4008-9b71-4267a04b8446 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:01.806150 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 121/251] 10.222.0.70 () {56 vars in 1101 bytes} [Thu Oct 27 09:24:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:24:01.811833 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd508d02-ad15-4614-ba7f-17767d5bdc8c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:01.840335 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:01.849282 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2a05c2-58dc-4738-9dd9-b48d78d7647d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:01.849282 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a05c2-58dc-4738-9dd9-b48d78d7647d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:01.849282 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a05c2-58dc-4738-9dd9-b48d78d7647d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:01.849282 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a05c2-58dc-4738-9dd9-b48d78d7647d None admin] PATH_INFO: `/v3/users/567a0a16618844dda14ddbd64da73c4b` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:01.850321 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2a05c2-58dc-4738-9dd9-b48d78d7647d None admin] RBAC: Authorizing `identity:get_user(user_id=567a0a16618844dda14ddbd64da73c4b)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:01.851775 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2a05c2-58dc-4738-9dd9-b48d78d7647d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:01.855636 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 131/252] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Oct 27 09:24:01 2022] GET /identity/v3/users/567a0a16618844dda14ddbd64da73c4b => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:24:01.859867 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2a05c2-58dc-4738-9dd9-b48d78d7647d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:01.878013 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:01.952966 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd508d02-ad15-4614-ba7f-17767d5bdc8c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:02.003769 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2a05c2-58dc-4738-9dd9-b48d78d7647d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:02.007159 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727a6333-f342-4d33-94de-510a5b0a70f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:02.008056 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a6333-f342-4d33-94de-510a5b0a70f7 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:02.009114 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a6333-f342-4d33-94de-510a5b0a70f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:02.009605 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a6333-f342-4d33-94de-510a5b0a70f7 None admin] PATH_INFO: `/v3/users/5625732a380241698f525db7052bab5a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:02.010963 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727a6333-f342-4d33-94de-510a5b0a70f7 None admin] RBAC: Authorizing `identity:get_user(user_id=5625732a380241698f525db7052bab5a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:02.013107 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727a6333-f342-4d33-94de-510a5b0a70f7 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:02.015647 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 122/253] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Oct 27 09:24:01 2022] GET /identity/v3/users/5625732a380241698f525db7052bab5a => generated 293 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:24:02.022308 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727a6333-f342-4d33-94de-510a5b0a70f7 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:02.068870 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5565e3a0-9dfb-40d7-a5e6-c051bba804b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:02.069533 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5565e3a0-9dfb-40d7-a5e6-c051bba804b4 None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:02.069811 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5565e3a0-9dfb-40d7-a5e6-c051bba804b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:02.070086 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5565e3a0-9dfb-40d7-a5e6-c051bba804b4 None admin] PATH_INFO: `/v3/system/users/567a0a16618844dda14ddbd64da73c4b/roles/f1938eac464c4f22bb492ea2fdf4899a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:02.075571 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5565e3a0-9dfb-40d7-a5e6-c051bba804b4 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=567a0a16618844dda14ddbd64da73c4b, role_id=f1938eac464c4f22bb492ea2fdf4899a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:02.077037 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5565e3a0-9dfb-40d7-a5e6-c051bba804b4 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:02.105110 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 132/254] 10.222.0.70 () {58 vars in 1334 bytes} [Thu Oct 27 09:24:01 2022] PUT /identity/v3/system/users/567a0a16618844dda14ddbd64da73c4b/roles/f1938eac464c4f22bb492ea2fdf4899a => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:24:02.150914 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:02.161304 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727a6333-f342-4d33-94de-510a5b0a70f7 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:02.203940 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1c0689-821a-4037-9457-2bb45613100b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:02.204489 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1c0689-821a-4037-9457-2bb45613100b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:02.204797 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1c0689-821a-4037-9457-2bb45613100b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:02.205067 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1c0689-821a-4037-9457-2bb45613100b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:02.205801 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1c0689-821a-4037-9457-2bb45613100b None admin] RBAC: Adding query filter params (role.id=eea7cd23e01b4506a9eb6ab86f9adcd2, user.id=5625732a380241698f525db7052bab5a, scope.system=all) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:02.206114 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1c0689-821a-4037-9457-2bb45613100b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:02.207575 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1c0689-821a-4037-9457-2bb45613100b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:02.225735 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 123/255] 10.222.0.70 () {56 vars in 1308 bytes} [Thu Oct 27 09:24:02 2022] GET /identity/v3/role_assignments?user.id=5625732a380241698f525db7052bab5a&scope.system=all&role.id=eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 510 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:03.712440 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e74a58-a43e-415d-bbf9-4f477f6e6659 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:03.713017 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e74a58-a43e-415d-bbf9-4f477f6e6659 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:03.713017 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e74a58-a43e-415d-bbf9-4f477f6e6659 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:03.714583 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 133/256] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:03.720847 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd74da6-3795-49d3-8542-251ea45ac9ba None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:03.721359 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd74da6-3795-49d3-8542-251ea45ac9ba None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:03.721434 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd74da6-3795-49d3-8542-251ea45ac9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:03.744969 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-edd74da6-3795-49d3-8542-251ea45ac9ba None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:03.746210 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-edd74da6-3795-49d3-8542-251ea45ac9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:03.822196 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 124/257] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:03 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 27 09:24:03.860894 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a97cb1-6d3c-46c5-811e-be2393f2e92d None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:03.861198 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a97cb1-6d3c-46c5-811e-be2393f2e92d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:03.861451 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a97cb1-6d3c-46c5-811e-be2393f2e92d None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:03.862931 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 134/258] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:03.870276 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a8935-9392-40d4-8a5e-4a37724c9cfb None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:03.870592 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a8935-9392-40d4-8a5e-4a37724c9cfb None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:03.870835 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a8935-9392-40d4-8a5e-4a37724c9cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:03.894442 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-f56a8935-9392-40d4-8a5e-4a37724c9cfb None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:03.895703 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f56a8935-9392-40d4-8a5e-4a37724c9cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:03.908097 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 125/259] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:03 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 27 09:24:04.157142 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a97cb1-6d3c-46c5-811e-be2393f2e92d None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:04.172888 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a97cb1-6d3c-46c5-811e-be2393f2e92d None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:04.177293 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2f2720-647a-427f-bf41-d404b7d1fe60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:04.177901 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f2720-647a-427f-bf41-d404b7d1fe60 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:04.178175 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f2720-647a-427f-bf41-d404b7d1fe60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:04.178446 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f2720-647a-427f-bf41-d404b7d1fe60 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:04.192308 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2f2720-647a-427f-bf41-d404b7d1fe60 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:24:04.201801 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-bf2f2720-647a-427f-bf41-d404b7d1fe60 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 27 09:24:04.201801 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:04.201801 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:04.201801 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:04.201801 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:04.201801 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:04.201801 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:04.201801 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:04.201801 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:04.201801 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:04.201801 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:04.201801 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:04.201801 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:24:04.201801 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:24:04.201801 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:24:04.201801 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:24:04.202658 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:24:04.202658 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:24:04.202658 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:24:04.202658 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:24:04.202658 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:04.202658 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:04.202658 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:04.202658 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:04.202658 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:04.202658 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:04.202658 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:04.202658 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:04.202658 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:04.202658 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:04.202658 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:04.202658 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:04.203476 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:04.203476 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:04.203476 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:04.203476 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:04.203476 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:24:04.203476 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:24:04.203476 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:24:04.203476 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:24:04.203476 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:24:04.203476 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:24:04.203476 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 27 09:24:04.203476 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:04.203476 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 135/260] 10.222.0.70 () {56 vars in 1120 bytes} [Thu Oct 27 09:24:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 46 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:04.208824 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a8935-9392-40d4-8a5e-4a37724c9cfb None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:04.221906 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a8935-9392-40d4-8a5e-4a37724c9cfb None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:04.232833 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0547316e-94a1-4584-ab3f-467b52303915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:04.233461 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547316e-94a1-4584-ab3f-467b52303915 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:04.233735 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547316e-94a1-4584-ab3f-467b52303915 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:04.233999 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547316e-94a1-4584-ab3f-467b52303915 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:04.234665 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0547316e-94a1-4584-ab3f-467b52303915 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:04.234963 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0547316e-94a1-4584-ab3f-467b52303915 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:04.236780 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: WARNING py.warnings [None req-0547316e-94a1-4584-ab3f-467b52303915 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 27 09:24:04.236780 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: warnings.warn(msg) Oct 27 09:24:04.236780 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: Oct 27 09:24:04.237667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0547316e-94a1-4584-ab3f-467b52303915 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:04.249030 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 126/261] 10.222.0.70 () {56 vars in 1122 bytes} [Thu Oct 27 09:24:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:04.254690 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2f2720-647a-427f-bf41-d404b7d1fe60 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:04.264922 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2f2720-647a-427f-bf41-d404b7d1fe60 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:04.276670 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882abb22-410b-4246-9934-7f0c892976c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:04.277321 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882abb22-410b-4246-9934-7f0c892976c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:04.277598 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882abb22-410b-4246-9934-7f0c892976c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:04.277869 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882abb22-410b-4246-9934-7f0c892976c7 admin admin] PATH_INFO: `/v3/groups/de7aaa46b2d64a949a760bbb17bf2523` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:04.288568 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882abb22-410b-4246-9934-7f0c892976c7 admin admin] RBAC: Authorizing `identity:get_group(group_id=de7aaa46b2d64a949a760bbb17bf2523)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:04.288568 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: WARNING py.warnings [None req-882abb22-410b-4246-9934-7f0c892976c7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 27 09:24:04.288568 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: warnings.warn(msg) Oct 27 09:24:04.288568 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: Oct 27 09:24:04.288568 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882abb22-410b-4246-9934-7f0c892976c7 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:04.294065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 136/262] 10.222.0.70 () {56 vars in 1201 bytes} [Thu Oct 27 09:24:04 2022] GET /identity/v3/groups/de7aaa46b2d64a949a760bbb17bf2523 => generated 226 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:04.299661 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0547316e-94a1-4584-ab3f-467b52303915 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:04.315004 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0547316e-94a1-4584-ab3f-467b52303915 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:04.329145 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e9a71-c10e-4064-9576-f184105d3a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:04.329795 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e9a71-c10e-4064-9576-f184105d3a6e admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:04.330075 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e9a71-c10e-4064-9576-f184105d3a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:04.330347 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e9a71-c10e-4064-9576-f184105d3a6e admin admin] PATH_INFO: `/v3/projects/78bd002b14ac4653ae0975911fcabf81` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:04.331538 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77e9a71-c10e-4064-9576-f184105d3a6e admin admin] RBAC: Authorizing `identity:get_project(project_id=78bd002b14ac4653ae0975911fcabf81)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:04.333682 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77e9a71-c10e-4064-9576-f184105d3a6e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:04.335336 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 127/263] 10.222.0.70 () {56 vars in 1207 bytes} [Thu Oct 27 09:24:04 2022] GET /identity/v3/projects/78bd002b14ac4653ae0975911fcabf81 => generated 298 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:04.341004 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882abb22-410b-4246-9934-7f0c892976c7 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:04.352624 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882abb22-410b-4246-9934-7f0c892976c7 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:04.361975 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008060cf-73d0-423b-9828-635cd111c883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:04.362697 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008060cf-73d0-423b-9828-635cd111c883 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:04.362973 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008060cf-73d0-423b-9828-635cd111c883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:04.363301 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008060cf-73d0-423b-9828-635cd111c883 admin admin] PATH_INFO: `/v3/projects/78bd002b14ac4653ae0975911fcabf81/groups/de7aaa46b2d64a949a760bbb17bf2523/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:04.369265 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008060cf-73d0-423b-9828-635cd111c883 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78bd002b14ac4653ae0975911fcabf81, group_id=de7aaa46b2d64a949a760bbb17bf2523, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:04.370284 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: WARNING py.warnings [None req-008060cf-73d0-423b-9828-635cd111c883 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 27 09:24:04.370284 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: warnings.warn(msg) Oct 27 09:24:04.370284 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: Oct 27 09:24:04.372236 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008060cf-73d0-423b-9828-635cd111c883 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:04.393368 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 137/264] 10.222.0.70 () {58 vars in 1463 bytes} [Thu Oct 27 09:24:04 2022] PUT /identity/v3/projects/78bd002b14ac4653ae0975911fcabf81/groups/de7aaa46b2d64a949a760bbb17bf2523/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:24:04.417774 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e9a71-c10e-4064-9576-f184105d3a6e admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:04.432137 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e9a71-c10e-4064-9576-f184105d3a6e admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:04.501586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a34c6eb-de5c-40ca-a42b-e0706b547de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:04.502680 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a34c6eb-de5c-40ca-a42b-e0706b547de5 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:04.503164 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a34c6eb-de5c-40ca-a42b-e0706b547de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:04.503685 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a34c6eb-de5c-40ca-a42b-e0706b547de5 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:04.516599 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a34c6eb-de5c-40ca-a42b-e0706b547de5 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:24:04.522130 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-4a34c6eb-de5c-40ca-a42b-e0706b547de5 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 27 09:24:04.522130 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:04.522130 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:04.522130 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:04.522130 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:04.522130 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:04.522130 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:04.522130 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:04.522130 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:04.522130 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:04.522130 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:04.522130 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:04.522130 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:24:04.522130 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:24:04.522130 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:24:04.522130 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:24:04.523040 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:24:04.523040 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:24:04.523040 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:24:04.523040 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:24:04.523040 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:04.523040 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:04.523040 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:04.523040 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:04.523040 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:04.523040 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:04.523040 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:04.523040 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:04.523040 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:04.523040 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:04.523040 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:04.523040 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:04.523867 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:04.523867 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:04.523867 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:04.523867 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:04.523867 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:24:04.523867 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:24:04.523867 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:24:04.523867 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:24:04.523867 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:24:04.523867 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:24:04.523867 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 27 09:24:04.523867 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:04.526525 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 128/265] 10.222.0.70 () {56 vars in 1135 bytes} [Thu Oct 27 09:24:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 109 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:04.537707 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008060cf-73d0-423b-9828-635cd111c883 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:04.555143 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008060cf-73d0-423b-9828-635cd111c883 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:04.560228 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eebd13-5918-4464-9467-c66d5962f956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:04.560897 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eebd13-5918-4464-9467-c66d5962f956 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:04.561176 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eebd13-5918-4464-9467-c66d5962f956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:04.561448 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eebd13-5918-4464-9467-c66d5962f956 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:04.562133 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eebd13-5918-4464-9467-c66d5962f956 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:04.562464 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eebd13-5918-4464-9467-c66d5962f956 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:04.563474 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: WARNING py.warnings [None req-e4eebd13-5918-4464-9467-c66d5962f956 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 27 09:24:04.563474 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: warnings.warn(msg) Oct 27 09:24:04.563474 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: Oct 27 09:24:04.564384 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eebd13-5918-4464-9467-c66d5962f956 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:04.578572 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 138/266] 10.222.0.70 () {56 vars in 1132 bytes} [Thu Oct 27 09:24:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:04.584687 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a34c6eb-de5c-40ca-a42b-e0706b547de5 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:04.600548 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a34c6eb-de5c-40ca-a42b-e0706b547de5 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:04.609836 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe4fa3-eeba-4b9c-ae0f-1623902474bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:04.610891 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe4fa3-eeba-4b9c-ae0f-1623902474bf admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:04.611407 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe4fa3-eeba-4b9c-ae0f-1623902474bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:04.611954 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe4fa3-eeba-4b9c-ae0f-1623902474bf admin admin] PATH_INFO: `/v3/groups/de7aaa46b2d64a949a760bbb17bf2523` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:04.617494 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbe4fa3-eeba-4b9c-ae0f-1623902474bf admin admin] RBAC: Authorizing `identity:get_group(group_id=de7aaa46b2d64a949a760bbb17bf2523)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:04.618598 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: WARNING py.warnings [None req-0fbe4fa3-eeba-4b9c-ae0f-1623902474bf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 27 09:24:04.618598 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: warnings.warn(msg) Oct 27 09:24:04.618598 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: Oct 27 09:24:04.619828 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbe4fa3-eeba-4b9c-ae0f-1623902474bf admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:04.621952 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 129/267] 10.222.0.70 () {56 vars in 1201 bytes} [Thu Oct 27 09:24:04 2022] GET /identity/v3/groups/de7aaa46b2d64a949a760bbb17bf2523 => generated 226 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:04.627919 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eebd13-5918-4464-9467-c66d5962f956 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:04.639331 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eebd13-5918-4464-9467-c66d5962f956 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:04.649790 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a87f34-1529-4a52-b6eb-066274a90be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:04.650447 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a87f34-1529-4a52-b6eb-066274a90be6 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:04.650729 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a87f34-1529-4a52-b6eb-066274a90be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:04.650998 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a87f34-1529-4a52-b6eb-066274a90be6 admin admin] PATH_INFO: `/v3/projects/78bd002b14ac4653ae0975911fcabf81` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:04.652140 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a87f34-1529-4a52-b6eb-066274a90be6 admin admin] RBAC: Authorizing `identity:get_project(project_id=78bd002b14ac4653ae0975911fcabf81)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:04.654246 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a87f34-1529-4a52-b6eb-066274a90be6 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:04.660775 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 139/268] 10.222.0.70 () {56 vars in 1207 bytes} [Thu Oct 27 09:24:04 2022] GET /identity/v3/projects/78bd002b14ac4653ae0975911fcabf81 => generated 298 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:04.662487 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe4fa3-eeba-4b9c-ae0f-1623902474bf admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:04.681295 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe4fa3-eeba-4b9c-ae0f-1623902474bf admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:04.685820 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6388701f-31be-479e-b68b-3ced00d97ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:04.686498 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388701f-31be-479e-b68b-3ced00d97ab1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:04.686774 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388701f-31be-479e-b68b-3ced00d97ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:04.687048 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388701f-31be-479e-b68b-3ced00d97ab1 admin admin] PATH_INFO: `/v3/projects/78bd002b14ac4653ae0975911fcabf81/groups/de7aaa46b2d64a949a760bbb17bf2523/roles/fa269b0e4dc54a24bb729398ce38231c` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:04.692389 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6388701f-31be-479e-b68b-3ced00d97ab1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78bd002b14ac4653ae0975911fcabf81, group_id=de7aaa46b2d64a949a760bbb17bf2523, role_id=fa269b0e4dc54a24bb729398ce38231c)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:04.693426 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: WARNING py.warnings [None req-6388701f-31be-479e-b68b-3ced00d97ab1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 27 09:24:04.693426 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: warnings.warn(msg) Oct 27 09:24:04.693426 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: Oct 27 09:24:04.695357 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6388701f-31be-479e-b68b-3ced00d97ab1 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:04.715071 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:04.720847 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 130/269] 10.222.0.70 () {58 vars in 1463 bytes} [Thu Oct 27 09:24:04 2022] PUT /identity/v3/projects/78bd002b14ac4653ae0975911fcabf81/groups/de7aaa46b2d64a949a760bbb17bf2523/roles/fa269b0e4dc54a24bb729398ce38231c => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:24:04.790843 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33289615-81a3-4a64-b66b-c8ea680bbf0f None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:04.791379 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33289615-81a3-4a64-b66b-c8ea680bbf0f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:04.791379 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33289615-81a3-4a64-b66b-c8ea680bbf0f None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:04.792735 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 140/270] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:04.799836 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0814441-7900-4fca-83a7-ae5260b5cf52 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:04.800311 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0814441-7900-4fca-83a7-ae5260b5cf52 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:04.800386 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0814441-7900-4fca-83a7-ae5260b5cf52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:04.820123 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a0814441-7900-4fca-83a7-ae5260b5cf52 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:04.821428 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a0814441-7900-4fca-83a7-ae5260b5cf52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:04.908737 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 131/271] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:04 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 27 09:24:05.190176 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33289615-81a3-4a64-b66b-c8ea680bbf0f None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:05.203239 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33289615-81a3-4a64-b66b-c8ea680bbf0f None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:05.208957 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074030df-c6b3-402a-b8da-8cd0110c936b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:05.209548 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074030df-c6b3-402a-b8da-8cd0110c936b admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:05.209822 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074030df-c6b3-402a-b8da-8cd0110c936b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:05.210089 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074030df-c6b3-402a-b8da-8cd0110c936b admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:05.222213 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074030df-c6b3-402a-b8da-8cd0110c936b admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:24:05.227803 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-074030df-c6b3-402a-b8da-8cd0110c936b admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 27 09:24:05.227803 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:05.227803 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:05.227803 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:05.227803 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:05.227803 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:05.227803 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:05.227803 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:05.227803 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:05.227803 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:05.227803 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:05.227803 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:05.227803 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:24:05.227803 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:24:05.227803 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:24:05.227803 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:24:05.228690 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:24:05.228690 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:24:05.228690 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:24:05.228690 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:24:05.228690 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:05.228690 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:05.228690 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:05.228690 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:05.228690 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:05.228690 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:05.228690 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:05.228690 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:05.228690 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:05.228690 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:05.228690 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:05.228690 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:05.229501 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:05.229501 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:05.229501 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:05.229501 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:05.229501 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:24:05.229501 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:24:05.229501 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:24:05.229501 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:24:05.229501 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:24:05.229501 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:24:05.229501 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 27 09:24:05.229501 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:05.232627 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 141/272] 10.222.0.70 () {56 vars in 1120 bytes} [Thu Oct 27 09:24:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 40 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:05.235421 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0814441-7900-4fca-83a7-ae5260b5cf52 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:05.244762 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0814441-7900-4fca-83a7-ae5260b5cf52 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:05.249545 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a9854-f3dc-484b-909b-0893a5a669d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:05.250216 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a9854-f3dc-484b-909b-0893a5a669d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:05.250441 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a9854-f3dc-484b-909b-0893a5a669d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:05.250710 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a9854-f3dc-484b-909b-0893a5a669d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:05.251375 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420a9854-f3dc-484b-909b-0893a5a669d1 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:05.251676 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420a9854-f3dc-484b-909b-0893a5a669d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:05.253377 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420a9854-f3dc-484b-909b-0893a5a669d1 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:05.267418 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 132/273] 10.222.0.70 () {56 vars in 1122 bytes} [Thu Oct 27 09:24:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:05.270735 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074030df-c6b3-402a-b8da-8cd0110c936b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:05.281131 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074030df-c6b3-402a-b8da-8cd0110c936b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:05.288563 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb03638c-6f06-40db-a130-1f973fb629d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:05.288563 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03638c-6f06-40db-a130-1f973fb629d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:05.288563 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03638c-6f06-40db-a130-1f973fb629d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:05.288563 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03638c-6f06-40db-a130-1f973fb629d5 admin admin] PATH_INFO: `/v3/groups/de7aaa46b2d64a949a760bbb17bf2523` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:05.288917 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb03638c-6f06-40db-a130-1f973fb629d5 admin admin] RBAC: Authorizing `identity:get_group(group_id=de7aaa46b2d64a949a760bbb17bf2523)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:05.288917 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb03638c-6f06-40db-a130-1f973fb629d5 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:05.294662 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 142/274] 10.222.0.70 () {56 vars in 1201 bytes} [Thu Oct 27 09:24:05 2022] GET /identity/v3/groups/de7aaa46b2d64a949a760bbb17bf2523 => generated 226 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:05.296036 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a9854-f3dc-484b-909b-0893a5a669d1 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:05.306158 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a9854-f3dc-484b-909b-0893a5a669d1 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:05.310585 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa315f6-b501-4dbf-8c31-adc7cda9e6f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:05.311218 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa315f6-b501-4dbf-8c31-adc7cda9e6f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:05.311497 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa315f6-b501-4dbf-8c31-adc7cda9e6f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:05.311759 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa315f6-b501-4dbf-8c31-adc7cda9e6f5 admin admin] PATH_INFO: `/v3/projects/e6e4e23ca6ee4fc0a033668401d7ba30` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:05.313231 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa315f6-b501-4dbf-8c31-adc7cda9e6f5 admin admin] RBAC: Authorizing `identity:get_project(project_id=e6e4e23ca6ee4fc0a033668401d7ba30)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:05.315425 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa315f6-b501-4dbf-8c31-adc7cda9e6f5 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:05.321171 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 133/275] 10.222.0.70 () {56 vars in 1207 bytes} [Thu Oct 27 09:24:05 2022] GET /identity/v3/projects/e6e4e23ca6ee4fc0a033668401d7ba30 => generated 302 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:05.323656 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb03638c-6f06-40db-a130-1f973fb629d5 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:05.335204 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb03638c-6f06-40db-a130-1f973fb629d5 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:05.340436 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34143b8b-cc14-42f2-853b-bf185b3a227b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:05.341130 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34143b8b-cc14-42f2-853b-bf185b3a227b admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:05.341408 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34143b8b-cc14-42f2-853b-bf185b3a227b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:05.341680 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34143b8b-cc14-42f2-853b-bf185b3a227b admin admin] PATH_INFO: `/v3/projects/e6e4e23ca6ee4fc0a033668401d7ba30/groups/de7aaa46b2d64a949a760bbb17bf2523/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:05.343559 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34143b8b-cc14-42f2-853b-bf185b3a227b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6e4e23ca6ee4fc0a033668401d7ba30, group_id=de7aaa46b2d64a949a760bbb17bf2523, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:05.346710 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34143b8b-cc14-42f2-853b-bf185b3a227b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:05.364798 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 143/276] 10.222.0.70 () {58 vars in 1463 bytes} [Thu Oct 27 09:24:05 2022] PUT /identity/v3/projects/e6e4e23ca6ee4fc0a033668401d7ba30/groups/de7aaa46b2d64a949a760bbb17bf2523/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:24:06.085303 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b56e92e-6e37-44fc-8c2a-3ea0ff35b6ab None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:06.085816 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b56e92e-6e37-44fc-8c2a-3ea0ff35b6ab None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:06.085893 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b56e92e-6e37-44fc-8c2a-3ea0ff35b6ab None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:06.087750 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 134/277] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:06.093439 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa8b9f0-5936-4457-bdbd-40bd6e1c4bb0 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:06.093936 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa8b9f0-5936-4457-bdbd-40bd6e1c4bb0 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:06.094011 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa8b9f0-5936-4457-bdbd-40bd6e1c4bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:06.102707 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:06.121454 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-baa8b9f0-5936-4457-bdbd-40bd6e1c4bb0 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:06.123274 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-baa8b9f0-5936-4457-bdbd-40bd6e1c4bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:06.197791 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 144/278] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:06 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 27 09:24:06.240189 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b87ecaa-4602-4f61-bb0d-5aab32f0fa3f None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:06.240504 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b87ecaa-4602-4f61-bb0d-5aab32f0fa3f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:06.240773 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b87ecaa-4602-4f61-bb0d-5aab32f0fa3f None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:06.242149 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 135/279] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:06.248510 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b40353-e61f-4b3f-80a5-44021801eed3 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:06.248833 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b40353-e61f-4b3f-80a5-44021801eed3 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:06.249097 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b40353-e61f-4b3f-80a5-44021801eed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:06.272485 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-74b40353-e61f-4b3f-80a5-44021801eed3 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:06.276552 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-74b40353-e61f-4b3f-80a5-44021801eed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:06.362339 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 145/280] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 27 09:24:06.368052 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d41e3ac-456b-475a-b350-fe2cd10dca94 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:06.368334 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d41e3ac-456b-475a-b350-fe2cd10dca94 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:06.368626 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d41e3ac-456b-475a-b350-fe2cd10dca94 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:06.370232 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 136/281] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:06.381833 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86dd2e-25d4-4d8e-b866-def1d81860c3 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:06.382114 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86dd2e-25d4-4d8e-b866-def1d81860c3 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:06.382391 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86dd2e-25d4-4d8e-b866-def1d81860c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:06.410449 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-4a86dd2e-25d4-4d8e-b866-def1d81860c3 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:06.411646 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4a86dd2e-25d4-4d8e-b866-def1d81860c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:06.425953 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 146/282] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:06 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 27 09:24:06.622252 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d41e3ac-456b-475a-b350-fe2cd10dca94 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:06.635883 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d41e3ac-456b-475a-b350-fe2cd10dca94 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:06.641057 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a59a09-1376-4120-85a4-ed223d38e473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:06.641689 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a59a09-1376-4120-85a4-ed223d38e473 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:06.641966 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a59a09-1376-4120-85a4-ed223d38e473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:06.642233 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a59a09-1376-4120-85a4-ed223d38e473 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:06.657232 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a59a09-1376-4120-85a4-ed223d38e473 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:24:06.662944 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-c1a59a09-1376-4120-85a4-ed223d38e473 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 27 09:24:06.662944 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:06.662944 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:06.662944 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:06.662944 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:06.662944 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:06.662944 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:06.662944 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:06.662944 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:06.662944 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:06.662944 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:06.662944 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:06.662944 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:24:06.662944 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:24:06.662944 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:24:06.662944 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:24:06.663816 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:24:06.663816 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:24:06.663816 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:24:06.663816 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:24:06.663816 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:06.663816 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:06.663816 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:06.663816 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:06.663816 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:06.663816 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:06.663816 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:06.663816 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:06.663816 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:06.663816 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:06.663816 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:06.663816 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:06.664681 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:06.664681 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:06.664681 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:06.664681 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:06.664681 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:24:06.664681 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:24:06.664681 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:24:06.664681 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:24:06.664681 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:24:06.664681 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:24:06.664681 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 27 09:24:06.664681 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:06.665344 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 137/283] 10.222.0.70 () {56 vars in 1135 bytes} [Thu Oct 27 09:24:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 43 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:06.669746 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86dd2e-25d4-4d8e-b866-def1d81860c3 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:06.685620 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86dd2e-25d4-4d8e-b866-def1d81860c3 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:06.701231 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f440cf-6f4d-4f12-bd8d-6fbcff63a3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:06.701862 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f440cf-6f4d-4f12-bd8d-6fbcff63a3c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:06.702137 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f440cf-6f4d-4f12-bd8d-6fbcff63a3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:06.702403 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f440cf-6f4d-4f12-bd8d-6fbcff63a3c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:06.703072 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f440cf-6f4d-4f12-bd8d-6fbcff63a3c5 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:06.703372 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f440cf-6f4d-4f12-bd8d-6fbcff63a3c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:06.704977 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f440cf-6f4d-4f12-bd8d-6fbcff63a3c5 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:06.714514 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 147/284] 10.222.0.70 () {56 vars in 1132 bytes} [Thu Oct 27 09:24:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:06.720306 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a59a09-1376-4120-85a4-ed223d38e473 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:06.734712 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a59a09-1376-4120-85a4-ed223d38e473 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:06.744904 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf264a-50a2-43e3-bcda-4876f53b8eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:06.745552 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf264a-50a2-43e3-bcda-4876f53b8eee admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:06.746013 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf264a-50a2-43e3-bcda-4876f53b8eee admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:06.746287 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf264a-50a2-43e3-bcda-4876f53b8eee admin admin] PATH_INFO: `/v3/groups/de7aaa46b2d64a949a760bbb17bf2523` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:06.747409 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf264a-50a2-43e3-bcda-4876f53b8eee admin admin] RBAC: Authorizing `identity:get_group(group_id=de7aaa46b2d64a949a760bbb17bf2523)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:06.749512 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf264a-50a2-43e3-bcda-4876f53b8eee admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:06.751081 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 138/285] 10.222.0.70 () {56 vars in 1201 bytes} [Thu Oct 27 09:24:06 2022] GET /identity/v3/groups/de7aaa46b2d64a949a760bbb17bf2523 => generated 226 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:06.756693 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f440cf-6f4d-4f12-bd8d-6fbcff63a3c5 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:06.769799 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f440cf-6f4d-4f12-bd8d-6fbcff63a3c5 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:06.780666 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e1f66-e16c-4a38-a39a-f937901efe27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:06.781312 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e1f66-e16c-4a38-a39a-f937901efe27 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:06.781584 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e1f66-e16c-4a38-a39a-f937901efe27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:06.781850 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e1f66-e16c-4a38-a39a-f937901efe27 admin admin] PATH_INFO: `/v3/projects/e6e4e23ca6ee4fc0a033668401d7ba30` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:06.789215 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373e1f66-e16c-4a38-a39a-f937901efe27 admin admin] RBAC: Authorizing `identity:get_project(project_id=e6e4e23ca6ee4fc0a033668401d7ba30)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:06.791299 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373e1f66-e16c-4a38-a39a-f937901efe27 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:06.792970 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 148/286] 10.222.0.70 () {56 vars in 1207 bytes} [Thu Oct 27 09:24:06 2022] GET /identity/v3/projects/e6e4e23ca6ee4fc0a033668401d7ba30 => generated 302 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:06.799802 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf264a-50a2-43e3-bcda-4876f53b8eee admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:06.813843 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf264a-50a2-43e3-bcda-4876f53b8eee admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:06.820100 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4bdbe-47c4-467e-b78c-543af56662df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:06.821343 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4bdbe-47c4-467e-b78c-543af56662df admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:06.821621 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4bdbe-47c4-467e-b78c-543af56662df admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:06.821891 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4bdbe-47c4-467e-b78c-543af56662df admin admin] PATH_INFO: `/v3/projects/e6e4e23ca6ee4fc0a033668401d7ba30/groups/de7aaa46b2d64a949a760bbb17bf2523/roles/fa269b0e4dc54a24bb729398ce38231c` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:06.822229 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e1f66-e16c-4a38-a39a-f937901efe27 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:06.824154 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a4bdbe-47c4-467e-b78c-543af56662df admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6e4e23ca6ee4fc0a033668401d7ba30, group_id=de7aaa46b2d64a949a760bbb17bf2523, role_id=fa269b0e4dc54a24bb729398ce38231c)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:06.827086 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a4bdbe-47c4-467e-b78c-543af56662df admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:06.847440 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 139/287] 10.222.0.70 () {58 vars in 1463 bytes} [Thu Oct 27 09:24:06 2022] PUT /identity/v3/projects/e6e4e23ca6ee4fc0a033668401d7ba30/groups/de7aaa46b2d64a949a760bbb17bf2523/roles/fa269b0e4dc54a24bb729398ce38231c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:24:06.909073 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4bdbe-47c4-467e-b78c-543af56662df admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:06.928906 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:07.020132 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4bdbe-47c4-467e-b78c-543af56662df admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:07.028274 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e1f66-e16c-4a38-a39a-f937901efe27 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:07.044026 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de235c-4e0a-48fb-a136-204b0d856b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:07.044709 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de235c-4e0a-48fb-a136-204b0d856b5b admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:07.044985 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de235c-4e0a-48fb-a136-204b0d856b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:07.045248 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de235c-4e0a-48fb-a136-204b0d856b5b admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:07.067136 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0de235c-4e0a-48fb-a136-204b0d856b5b admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:24:07.072788 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-b0de235c-4e0a-48fb-a136-204b0d856b5b admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 09:24:07.072788 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:07.072788 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:07.072788 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:07.072788 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:07.072788 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:07.072788 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:07.072788 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:07.072788 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:07.072788 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:07.072788 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:07.072788 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:07.072788 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:24:07.072788 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:24:07.072788 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:24:07.072788 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:24:07.073699 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:24:07.073699 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:24:07.073699 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:24:07.073699 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:24:07.073699 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:07.073699 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:07.073699 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:07.073699 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:07.073699 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:07.073699 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:07.073699 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:07.073699 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:07.073699 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:07.073699 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:07.073699 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:07.073699 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:07.074487 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:07.074487 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:07.074487 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:07.074487 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:07.074487 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:24:07.074487 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:24:07.074487 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:24:07.074487 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:24:07.074487 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:24:07.074487 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:24:07.074487 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 09:24:07.074487 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:07.075560 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 140/288] 10.222.0.70 () {56 vars in 1117 bytes} [Thu Oct 27 09:24:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 09:24:07.085916 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de235c-4e0a-48fb-a136-204b0d856b5b admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:07.107770 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de235c-4e0a-48fb-a136-204b0d856b5b admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:07.118078 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c23c7d2-3ef2-4e75-b272-039845729a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:07.118684 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23c7d2-3ef2-4e75-b272-039845729a36 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:07.118957 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23c7d2-3ef2-4e75-b272-039845729a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:07.119221 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23c7d2-3ef2-4e75-b272-039845729a36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:07.119883 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c23c7d2-3ef2-4e75-b272-039845729a36 admin admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:07.120177 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c23c7d2-3ef2-4e75-b272-039845729a36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:07.123487 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ffcafd-88d5-4428-8c9c-c6f9f8b5cfe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:07.124094 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ffcafd-88d5-4428-8c9c-c6f9f8b5cfe9 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:07.124375 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ffcafd-88d5-4428-8c9c-c6f9f8b5cfe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:07.124666 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ffcafd-88d5-4428-8c9c-c6f9f8b5cfe9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:07.127677 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c23c7d2-3ef2-4e75-b272-039845729a36 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:07.142086 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ffcafd-88d5-4428-8c9c-c6f9f8b5cfe9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:24:07.147370 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-05ffcafd-88d5-4428-8c9c-c6f9f8b5cfe9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 27 09:24:07.147370 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:07.147370 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:07.147370 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:07.147370 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:07.147370 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:07.147370 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:07.147370 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:07.147370 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:07.147370 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:07.147370 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:07.147370 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:07.147370 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:24:07.147370 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:24:07.147370 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:24:07.147370 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:24:07.149787 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:24:07.149787 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:24:07.149787 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:24:07.149787 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:24:07.149787 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:07.149787 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:07.149787 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:07.149787 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:07.149787 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:07.149787 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:07.149787 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:07.149787 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:07.149787 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:07.149787 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:07.149787 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:07.149787 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:07.151750 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:07.151750 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:07.151750 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:07.151750 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:07.151750 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:24:07.151750 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:24:07.151750 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:24:07.151750 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:24:07.151750 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:24:07.151750 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:24:07.151750 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 27 09:24:07.151750 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:07.151750 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 149/289] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Oct 27 09:24:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 09:24:07.157164 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 141/290] 10.222.0.70 () {56 vars in 1120 bytes} [Thu Oct 27 09:24:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:07.157401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ffcafd-88d5-4428-8c9c-c6f9f8b5cfe9 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:07.159030 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c23c7d2-3ef2-4e75-b272-039845729a36 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:07.170576 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ffcafd-88d5-4428-8c9c-c6f9f8b5cfe9 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:07.181255 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae6e261-7abb-4bc7-8099-e9117e5fbf36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:07.182145 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6e261-7abb-4bc7-8099-e9117e5fbf36 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:07.182440 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6e261-7abb-4bc7-8099-e9117e5fbf36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:07.182703 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6e261-7abb-4bc7-8099-e9117e5fbf36 admin admin] PATH_INFO: `/v3/groups/7fea783eff8f47ddb9386a51f3485c77` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:07.194251 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae6e261-7abb-4bc7-8099-e9117e5fbf36 admin admin] RBAC: Authorizing `identity:get_group(group_id=7fea783eff8f47ddb9386a51f3485c77)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:07.199665 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae6e261-7abb-4bc7-8099-e9117e5fbf36 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:07.203652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 150/291] 10.222.0.70 () {56 vars in 1201 bytes} [Thu Oct 27 09:24:07 2022] GET /identity/v3/groups/7fea783eff8f47ddb9386a51f3485c77 => generated 229 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:07.210147 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae6e261-7abb-4bc7-8099-e9117e5fbf36 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:07.218939 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:07.230472 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae6e261-7abb-4bc7-8099-e9117e5fbf36 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:07.241339 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddfa482-db3c-4d20-b4a9-3cdb12fe7391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:07.241983 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddfa482-db3c-4d20-b4a9-3cdb12fe7391 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:07.244680 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddfa482-db3c-4d20-b4a9-3cdb12fe7391 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:07.244968 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddfa482-db3c-4d20-b4a9-3cdb12fe7391 admin admin] PATH_INFO: `/v3/projects/d2f8e01e4d9d43978ddeb5033ead57f5` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:07.249920 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddfa482-db3c-4d20-b4a9-3cdb12fe7391 admin admin] RBAC: Authorizing `identity:get_project(project_id=d2f8e01e4d9d43978ddeb5033ead57f5)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:07.251812 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddfa482-db3c-4d20-b4a9-3cdb12fe7391 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:07.256507 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 151/292] 10.222.0.70 () {56 vars in 1207 bytes} [Thu Oct 27 09:24:07 2022] GET /identity/v3/projects/d2f8e01e4d9d43978ddeb5033ead57f5 => generated 344 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:07.262351 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddfa482-db3c-4d20-b4a9-3cdb12fe7391 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:07.278258 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddfa482-db3c-4d20-b4a9-3cdb12fe7391 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:07.290401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882b9bae-0eab-4dcc-9f4c-991cde4f68f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:07.291270 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b9bae-0eab-4dcc-9f4c-991cde4f68f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:07.291548 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b9bae-0eab-4dcc-9f4c-991cde4f68f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:07.291818 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b9bae-0eab-4dcc-9f4c-991cde4f68f2 admin admin] PATH_INFO: `/v3/projects/d2f8e01e4d9d43978ddeb5033ead57f5/groups/7fea783eff8f47ddb9386a51f3485c77/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:07.296834 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882b9bae-0eab-4dcc-9f4c-991cde4f68f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2f8e01e4d9d43978ddeb5033ead57f5, group_id=7fea783eff8f47ddb9386a51f3485c77, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:07.303568 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882b9bae-0eab-4dcc-9f4c-991cde4f68f2 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:07.334243 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 152/293] 10.222.0.70 () {58 vars in 1463 bytes} [Thu Oct 27 09:24:07 2022] PUT /identity/v3/projects/d2f8e01e4d9d43978ddeb5033ead57f5/groups/7fea783eff8f47ddb9386a51f3485c77/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:24:07.371357 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c23c7d2-3ef2-4e75-b272-039845729a36 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:07.424305 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc841c00-583e-4a32-9694-5547cc912d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:07.424889 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc841c00-583e-4a32-9694-5547cc912d44 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:07.425164 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc841c00-583e-4a32-9694-5547cc912d44 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:07.425429 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc841c00-583e-4a32-9694-5547cc912d44 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:07.426089 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc841c00-583e-4a32-9694-5547cc912d44 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:07.426392 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc841c00-583e-4a32-9694-5547cc912d44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:07.427758 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc841c00-583e-4a32-9694-5547cc912d44 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:07.440436 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 142/294] 10.222.0.70 () {56 vars in 1101 bytes} [Thu Oct 27 09:24:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:07.446186 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882b9bae-0eab-4dcc-9f4c-991cde4f68f2 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:07.578418 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:07.589536 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882b9bae-0eab-4dcc-9f4c-991cde4f68f2 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:07.629372 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a691457-f307-4e27-b093-7b0b51a794a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:07.630099 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a691457-f307-4e27-b093-7b0b51a794a3 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:07.630375 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a691457-f307-4e27-b093-7b0b51a794a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:07.630645 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a691457-f307-4e27-b093-7b0b51a794a3 None admin] PATH_INFO: `/v3/users/567a0a16618844dda14ddbd64da73c4b` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:07.631929 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a691457-f307-4e27-b093-7b0b51a794a3 None admin] RBAC: Authorizing `identity:get_user(user_id=567a0a16618844dda14ddbd64da73c4b)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:07.633544 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a691457-f307-4e27-b093-7b0b51a794a3 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:07.635257 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 153/295] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Oct 27 09:24:07 2022] GET /identity/v3/users/567a0a16618844dda14ddbd64da73c4b => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:07.641302 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc841c00-583e-4a32-9694-5547cc912d44 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:07.755549 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:07.764063 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc841c00-583e-4a32-9694-5547cc912d44 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:07.808855 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee74dec8-7a19-4045-a052-67d9dc0018d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:07.809826 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee74dec8-7a19-4045-a052-67d9dc0018d4 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:07.810318 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee74dec8-7a19-4045-a052-67d9dc0018d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:07.810759 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee74dec8-7a19-4045-a052-67d9dc0018d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:07.811660 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee74dec8-7a19-4045-a052-67d9dc0018d4 None admin] RBAC: Adding query filter params (role.id=f1938eac464c4f22bb492ea2fdf4899a, user.id=567a0a16618844dda14ddbd64da73c4b, scope.system=all) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:07.812138 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee74dec8-7a19-4045-a052-67d9dc0018d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:07.813796 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee74dec8-7a19-4045-a052-67d9dc0018d4 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:07.827256 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 143/296] 10.222.0.70 () {56 vars in 1308 bytes} [Thu Oct 27 09:24:07 2022] GET /identity/v3/role_assignments?user.id=567a0a16618844dda14ddbd64da73c4b&scope.system=all&role.id=f1938eac464c4f22bb492ea2fdf4899a => generated 510 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:07.911966 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6160e9d-0e4e-4adc-a013-21225f782089 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:07.912611 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6160e9d-0e4e-4adc-a013-21225f782089 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:07.913064 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6160e9d-0e4e-4adc-a013-21225f782089 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:07.915945 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 154/297] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:07.921333 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b244f-0430-4a62-b9dd-e53206c10c9f None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:07.921826 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b244f-0430-4a62-b9dd-e53206c10c9f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:07.922270 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b244f-0430-4a62-b9dd-e53206c10c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:07.946994 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-ca6b244f-0430-4a62-b9dd-e53206c10c9f None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:07.948506 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ca6b244f-0430-4a62-b9dd-e53206c10c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:08.032141 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 144/298] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:24:08.358766 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6160e9d-0e4e-4adc-a013-21225f782089 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:08.478100 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6160e9d-0e4e-4adc-a013-21225f782089 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:08.515960 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3794d6-ffe7-4b87-82cd-56ec6ec981c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:08.516616 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3794d6-ffe7-4b87-82cd-56ec6ec981c2 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:08.516893 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3794d6-ffe7-4b87-82cd-56ec6ec981c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:08.517157 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3794d6-ffe7-4b87-82cd-56ec6ec981c2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:08.527483 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3794d6-ffe7-4b87-82cd-56ec6ec981c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:24:08.532791 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-3d3794d6-ffe7-4b87-82cd-56ec6ec981c2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 27 09:24:08.532791 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:08.532791 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:08.532791 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:08.532791 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:08.532791 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:08.532791 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:08.532791 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:08.532791 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:08.532791 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:08.532791 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:08.532791 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:08.532791 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:24:08.532791 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:24:08.532791 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:24:08.532791 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:24:08.533672 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:24:08.533672 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:24:08.533672 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:24:08.533672 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:24:08.533672 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:08.533672 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:08.533672 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:08.533672 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:08.533672 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:08.533672 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:08.533672 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:08.533672 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:08.533672 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:08.533672 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:08.533672 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:08.533672 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:08.534915 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:08.534915 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:08.534915 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:08.534915 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:08.534915 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:24:08.534915 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:24:08.534915 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:24:08.534915 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:24:08.534915 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:24:08.534915 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:24:08.534915 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 27 09:24:08.534915 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:08.534915 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 155/299] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Oct 27 09:24:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:08.540143 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6b244f-0430-4a62-b9dd-e53206c10c9f None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:08.649979 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:08.671473 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6b244f-0430-4a62-b9dd-e53206c10c9f None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:08.713581 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad352f7-7a57-4d6d-bc32-9ebb5e5d1980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:08.714207 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad352f7-7a57-4d6d-bc32-9ebb5e5d1980 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:08.714476 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad352f7-7a57-4d6d-bc32-9ebb5e5d1980 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:08.714741 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad352f7-7a57-4d6d-bc32-9ebb5e5d1980 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:08.715418 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad352f7-7a57-4d6d-bc32-9ebb5e5d1980 None admin] RBAC: Adding query filter params (name=member) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:08.715718 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad352f7-7a57-4d6d-bc32-9ebb5e5d1980 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:08.717182 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad352f7-7a57-4d6d-bc32-9ebb5e5d1980 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:08.732312 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 145/300] 10.222.0.70 () {56 vars in 1101 bytes} [Thu Oct 27 09:24:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:08.738125 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3794d6-ffe7-4b87-82cd-56ec6ec981c2 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:08.769881 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:08.771652 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a622ab-862c-4451-8438-20f32ed2fec7 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:08.771989 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a622ab-862c-4451-8438-20f32ed2fec7 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:08.772275 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a622ab-862c-4451-8438-20f32ed2fec7 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:08.773963 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 146/301] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:08.802475 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07619ec2-1058-4fce-985c-eaebe1ff098e None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:08.802766 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07619ec2-1058-4fce-985c-eaebe1ff098e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:08.802950 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07619ec2-1058-4fce-985c-eaebe1ff098e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:08.829590 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-07619ec2-1058-4fce-985c-eaebe1ff098e None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:08.830844 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-07619ec2-1058-4fce-985c-eaebe1ff098e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:08.860796 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3794d6-ffe7-4b87-82cd-56ec6ec981c2 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:08.901437 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cdd9f9-a79d-4b63-a8b8-2b7852cdd598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:08.902059 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cdd9f9-a79d-4b63-a8b8-2b7852cdd598 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:08.902814 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cdd9f9-a79d-4b63-a8b8-2b7852cdd598 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:08.903089 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cdd9f9-a79d-4b63-a8b8-2b7852cdd598 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:08.911445 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 147/302] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:24:08.913049 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8599c7b-a84c-4ff9-8917-b8a3c59f3bb5 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:08.913489 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8599c7b-a84c-4ff9-8917-b8a3c59f3bb5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:08.913897 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8599c7b-a84c-4ff9-8917-b8a3c59f3bb5 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:08.916039 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 148/303] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:08.922186 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e267569-eb43-47c5-93a5-08ef3d7d543a None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:08.922636 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e267569-eb43-47c5-93a5-08ef3d7d543a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:08.923107 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e267569-eb43-47c5-93a5-08ef3d7d543a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:08.954345 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-9e267569-eb43-47c5-93a5-08ef3d7d543a None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:08.955528 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9e267569-eb43-47c5-93a5-08ef3d7d543a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:09.027726 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cdd9f9-a79d-4b63-a8b8-2b7852cdd598 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:09.029225 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cdd9f9-a79d-4b63-a8b8-2b7852cdd598 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:09.057686 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:09.111174 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 149/304] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 27 09:24:09.146106 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 156/305] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Oct 27 09:24:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 27 09:24:09.152238 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e267569-eb43-47c5-93a5-08ef3d7d543a None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:09.156204 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:09.260419 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e267569-eb43-47c5-93a5-08ef3d7d543a None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:09.320729 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae4b145-7729-4e85-bfa4-8c1bb9db557b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:09.321340 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae4b145-7729-4e85-bfa4-8c1bb9db557b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:09.321616 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae4b145-7729-4e85-bfa4-8c1bb9db557b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:09.321887 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae4b145-7729-4e85-bfa4-8c1bb9db557b None admin] PATH_INFO: `/v3/projects/78bd002b14ac4653ae0975911fcabf81` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:09.323013 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae4b145-7729-4e85-bfa4-8c1bb9db557b None admin] RBAC: Authorizing `identity:get_project(project_id=78bd002b14ac4653ae0975911fcabf81)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:09.324442 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae4b145-7729-4e85-bfa4-8c1bb9db557b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:09.327356 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 150/306] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Oct 27 09:24:09 2022] GET /identity/v3/projects/78bd002b14ac4653ae0975911fcabf81 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:09.332141 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cdd9f9-a79d-4b63-a8b8-2b7852cdd598 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:09.370720 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae4b145-7729-4e85-bfa4-8c1bb9db557b None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:09.475833 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cdd9f9-a79d-4b63-a8b8-2b7852cdd598 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:09.499310 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:09.501228 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:09.524866 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cd00da-0fb8-4d33-8a6a-b8a11fe67630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:09.525504 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd00da-0fb8-4d33-8a6a-b8a11fe67630 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:09.525782 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd00da-0fb8-4d33-8a6a-b8a11fe67630 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:09.526049 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd00da-0fb8-4d33-8a6a-b8a11fe67630 None admin] PATH_INFO: `/v3/groups/de7aaa46b2d64a949a760bbb17bf2523` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:09.527931 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cd00da-0fb8-4d33-8a6a-b8a11fe67630 None admin] RBAC: Authorizing `identity:get_group(group_id=de7aaa46b2d64a949a760bbb17bf2523)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:09.529392 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cd00da-0fb8-4d33-8a6a-b8a11fe67630 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:09.531265 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 157/307] 10.222.0.70 () {56 vars in 1180 bytes} [Thu Oct 27 09:24:09 2022] GET /identity/v3/groups/de7aaa46b2d64a949a760bbb17bf2523 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:09.537689 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cd00da-0fb8-4d33-8a6a-b8a11fe67630 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:09.558611 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae4b145-7729-4e85-bfa4-8c1bb9db557b None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:09.599552 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e471a6c-baa2-4b08-9214-8dd0d0589a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:09.600203 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e471a6c-baa2-4b08-9214-8dd0d0589a75 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:09.600487 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e471a6c-baa2-4b08-9214-8dd0d0589a75 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:09.600789 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e471a6c-baa2-4b08-9214-8dd0d0589a75 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:09.614073 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e471a6c-baa2-4b08-9214-8dd0d0589a75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:24:09.619949 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-0e471a6c-baa2-4b08-9214-8dd0d0589a75 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 27 09:24:09.619949 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:09.619949 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:09.619949 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:09.619949 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:09.619949 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:09.619949 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:09.619949 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:09.619949 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:09.619949 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:09.619949 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:09.619949 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:09.619949 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:24:09.619949 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:24:09.619949 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:24:09.619949 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:24:09.620869 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:24:09.620869 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:24:09.620869 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:24:09.620869 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:24:09.620869 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:09.620869 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:09.620869 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:09.620869 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:09.620869 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:09.620869 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:09.620869 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:09.620869 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:09.620869 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:09.620869 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:09.620869 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:09.620869 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:09.621670 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:09.621670 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:09.621670 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:09.621670 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:09.621670 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:24:09.621670 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:24:09.621670 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:24:09.621670 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:24:09.621670 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:24:09.621670 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:24:09.621670 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 27 09:24:09.621670 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:09.621670 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 151/308] 10.222.0.70 () {56 vars in 1114 bytes} [Thu Oct 27 09:24:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:09.622324 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e471a6c-baa2-4b08-9214-8dd0d0589a75 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:09.666539 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cd00da-0fb8-4d33-8a6a-b8a11fe67630 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:09.709210 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4a48cb-29ff-4666-a959-d4c9ea05c9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:09.709821 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a48cb-29ff-4666-a959-d4c9ea05c9c2 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:09.710096 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a48cb-29ff-4666-a959-d4c9ea05c9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:09.710362 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a48cb-29ff-4666-a959-d4c9ea05c9c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:09.711686 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4a48cb-29ff-4666-a959-d4c9ea05c9c2 None admin] RBAC: Adding query filter params (group.id=de7aaa46b2d64a949a760bbb17bf2523, role.id=eea7cd23e01b4506a9eb6ab86f9adcd2, scope.project.id=78bd002b14ac4653ae0975911fcabf81) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:09.712094 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4a48cb-29ff-4666-a959-d4c9ea05c9c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:09.714003 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4a48cb-29ff-4666-a959-d4c9ea05c9c2 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:09.722351 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 158/309] 10.222.0.70 () {56 vars in 1376 bytes} [Thu Oct 27 09:24:09 2022] GET /identity/v3/role_assignments?group.id=de7aaa46b2d64a949a760bbb17bf2523&scope.project.id=78bd002b14ac4653ae0975911fcabf81&role.id=eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 611 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:24:09.723174 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4a48cb-29ff-4666-a959-d4c9ea05c9c2 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:09.749008 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e471a6c-baa2-4b08-9214-8dd0d0589a75 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:09.794357 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2ae1a5-eb09-4745-bc62-0d19ba9564ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:09.795009 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ae1a5-eb09-4745-bc62-0d19ba9564ec None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:09.795291 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ae1a5-eb09-4745-bc62-0d19ba9564ec None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:09.795555 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ae1a5-eb09-4745-bc62-0d19ba9564ec None admin] PATH_INFO: `/v3/roles/member` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:09.799156 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:09.812242 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2ae1a5-eb09-4745-bc62-0d19ba9564ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:24:09.818215 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-0a2ae1a5-eb09-4745-bc62-0d19ba9564ec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 27 09:24:09.818215 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:09.818215 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:09.818215 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:09.818215 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:09.818215 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:09.818215 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:09.818215 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:09.818215 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:09.818215 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:09.818215 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:09.818215 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:09.818215 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:24:09.818215 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:24:09.818215 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:24:09.818215 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:24:09.819162 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:24:09.819162 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:24:09.819162 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:24:09.819162 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:24:09.819162 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:09.819162 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:09.819162 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:09.819162 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:09.819162 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:09.819162 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:09.819162 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:09.819162 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:09.819162 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:09.819162 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:09.819162 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:09.819162 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:09.819967 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:09.819967 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:09.819967 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:09.819967 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:09.819967 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:24:09.819967 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:24:09.819967 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:24:09.819967 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:24:09.819967 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:24:09.819967 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:24:09.819967 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 27 09:24:09.819967 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:09.819967 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 152/310] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Oct 27 09:24:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:09.829885 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2ae1a5-eb09-4745-bc62-0d19ba9564ec None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:09.869008 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:09.957657 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4a48cb-29ff-4666-a959-d4c9ea05c9c2 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:10.025682 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2ae1a5-eb09-4745-bc62-0d19ba9564ec None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:10.026801 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8881db4d-ac95-4537-a92b-6e06e9adaa56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:10.027767 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881db4d-ac95-4537-a92b-6e06e9adaa56 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:10.028045 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881db4d-ac95-4537-a92b-6e06e9adaa56 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:10.029392 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881db4d-ac95-4537-a92b-6e06e9adaa56 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:10.030120 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8881db4d-ac95-4537-a92b-6e06e9adaa56 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:10.030424 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8881db4d-ac95-4537-a92b-6e06e9adaa56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:10.031925 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8881db4d-ac95-4537-a92b-6e06e9adaa56 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:10.042671 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 159/311] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Oct 27 09:24:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:10.049066 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8881db4d-ac95-4537-a92b-6e06e9adaa56 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:10.080309 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe895d3-cf81-4f17-9cf9-d9289f97dab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:10.081533 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe895d3-cf81-4f17-9cf9-d9289f97dab0 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:10.081821 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe895d3-cf81-4f17-9cf9-d9289f97dab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:10.082088 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe895d3-cf81-4f17-9cf9-d9289f97dab0 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:10.083043 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe895d3-cf81-4f17-9cf9-d9289f97dab0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:10.083344 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe895d3-cf81-4f17-9cf9-d9289f97dab0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:10.085476 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe895d3-cf81-4f17-9cf9-d9289f97dab0 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:10.098433 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 153/312] 10.222.0.70 () {56 vars in 1101 bytes} [Thu Oct 27 09:24:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:10.104021 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe895d3-cf81-4f17-9cf9-d9289f97dab0 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:10.181625 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:10.190721 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8881db4d-ac95-4537-a92b-6e06e9adaa56 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:10.212013 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:10.240487 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1953b580-8c68-49d3-8758-4e8a0df6d362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:10.241135 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953b580-8c68-49d3-8758-4e8a0df6d362 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:10.241417 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953b580-8c68-49d3-8758-4e8a0df6d362 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:10.241692 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953b580-8c68-49d3-8758-4e8a0df6d362 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:10.245071 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe895d3-cf81-4f17-9cf9-d9289f97dab0 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:10.293702 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f490cc61-3010-453f-b0d9-c1497918222b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:10.294825 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f490cc61-3010-453f-b0d9-c1497918222b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:10.295306 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f490cc61-3010-453f-b0d9-c1497918222b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:10.296123 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f490cc61-3010-453f-b0d9-c1497918222b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:10.322580 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1953b580-8c68-49d3-8758-4e8a0df6d362 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:10.324130 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1953b580-8c68-49d3-8758-4e8a0df6d362 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:10.386891 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f490cc61-3010-453f-b0d9-c1497918222b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:10.388349 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f490cc61-3010-453f-b0d9-c1497918222b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:10.451807 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 160/313] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Oct 27 09:24:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 27 09:24:10.454839 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66a775-f44c-463c-8f43-2eb210a7eac4 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:10.454839 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66a775-f44c-463c-8f43-2eb210a7eac4 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:10.454839 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66a775-f44c-463c-8f43-2eb210a7eac4 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:10.460473 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 161/314] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:10.460473 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd66a775-f44c-463c-8f43-2eb210a7eac4 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:10.474155 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:10.479959 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:10.531685 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 154/315] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Oct 27 09:24:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 27 09:24:10.533323 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52777d9-07eb-480e-83e0-5471fa5f5d2d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:10.533821 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52777d9-07eb-480e-83e0-5471fa5f5d2d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:10.534293 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52777d9-07eb-480e-83e0-5471fa5f5d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:10.557763 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-c52777d9-07eb-480e-83e0-5471fa5f5d2d None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:10.559774 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c52777d9-07eb-480e-83e0-5471fa5f5d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:10.595882 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd66a775-f44c-463c-8f43-2eb210a7eac4 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:10.646433 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8cea71-8bd5-4d2d-8a37-03645c52d89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:10.647084 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8cea71-8bd5-4d2d-8a37-03645c52d89c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:10.647344 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8cea71-8bd5-4d2d-8a37-03645c52d89c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:10.647611 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8cea71-8bd5-4d2d-8a37-03645c52d89c None admin] PATH_INFO: `/v3/projects/78bd002b14ac4653ae0975911fcabf81` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:10.648125 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 155/316] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:24:10.649541 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52777d9-07eb-480e-83e0-5471fa5f5d2d None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:10.649820 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8cea71-8bd5-4d2d-8a37-03645c52d89c None admin] RBAC: Authorizing `identity:get_project(project_id=78bd002b14ac4653ae0975911fcabf81)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:10.651486 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8cea71-8bd5-4d2d-8a37-03645c52d89c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:10.653452 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 162/317] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Oct 27 09:24:10 2022] GET /identity/v3/projects/78bd002b14ac4653ae0975911fcabf81 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:10.660998 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8cea71-8bd5-4d2d-8a37-03645c52d89c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:10.763166 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:10.764718 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:10.784370 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8cea71-8bd5-4d2d-8a37-03645c52d89c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:10.786508 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52777d9-07eb-480e-83e0-5471fa5f5d2d None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:10.827015 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b848179-63e9-4cef-8e77-2ddaac723f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:10.827652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b848179-63e9-4cef-8e77-2ddaac723f52 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:10.827931 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b848179-63e9-4cef-8e77-2ddaac723f52 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:10.828200 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b848179-63e9-4cef-8e77-2ddaac723f52 None admin] PATH_INFO: `/v3/groups/de7aaa46b2d64a949a760bbb17bf2523` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:10.830220 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b848179-63e9-4cef-8e77-2ddaac723f52 None admin] RBAC: Authorizing `identity:get_group(group_id=de7aaa46b2d64a949a760bbb17bf2523)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:10.830788 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b848179-63e9-4cef-8e77-2ddaac723f52 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:10.832354 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 163/318] 10.222.0.70 () {56 vars in 1180 bytes} [Thu Oct 27 09:24:10 2022] GET /identity/v3/groups/de7aaa46b2d64a949a760bbb17bf2523 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:10.834531 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5700857-41ea-4e65-bbe5-c1d78fe383fa None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:10.834775 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5700857-41ea-4e65-bbe5-c1d78fe383fa None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:10.835052 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5700857-41ea-4e65-bbe5-c1d78fe383fa None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:10.835629 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb825cf-135c-4534-9a6f-f40d043c2ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:10.837020 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb825cf-135c-4534-9a6f-f40d043c2ee6 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:10.837145 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 164/319] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:10.837532 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb825cf-135c-4534-9a6f-f40d043c2ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:10.837927 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb825cf-135c-4534-9a6f-f40d043c2ee6 None admin] PATH_INFO: `/v3/projects/e6e4e23ca6ee4fc0a033668401d7ba30` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:10.840283 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb825cf-135c-4534-9a6f-f40d043c2ee6 None admin] RBAC: Authorizing `identity:get_project(project_id=e6e4e23ca6ee4fc0a033668401d7ba30)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:10.841815 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5700857-41ea-4e65-bbe5-c1d78fe383fa None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:10.842145 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb825cf-135c-4534-9a6f-f40d043c2ee6 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:10.843931 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 156/320] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Oct 27 09:24:10 2022] GET /identity/v3/projects/e6e4e23ca6ee4fc0a033668401d7ba30 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:10.845731 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c74969-4188-4176-b416-aef2f45cce5b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:10.846183 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c74969-4188-4176-b416-aef2f45cce5b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:10.847550 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c74969-4188-4176-b416-aef2f45cce5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:10.870506 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-d9c74969-4188-4176-b416-aef2f45cce5b None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:10.871688 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d9c74969-4188-4176-b416-aef2f45cce5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:10.953941 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 157/321] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:24:10.954795 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c74969-4188-4176-b416-aef2f45cce5b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:10.967071 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5700857-41ea-4e65-bbe5-c1d78fe383fa None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:11.010136 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b505fe6e-8ba5-46b5-b16e-3b0caf0b713e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:11.010907 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b505fe6e-8ba5-46b5-b16e-3b0caf0b713e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:11.011187 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b505fe6e-8ba5-46b5-b16e-3b0caf0b713e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:11.011456 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b505fe6e-8ba5-46b5-b16e-3b0caf0b713e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:11.012175 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b505fe6e-8ba5-46b5-b16e-3b0caf0b713e None admin] RBAC: Adding query filter params (group.id=de7aaa46b2d64a949a760bbb17bf2523, role.id=fa269b0e4dc54a24bb729398ce38231c, scope.project.id=78bd002b14ac4653ae0975911fcabf81) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:11.012482 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b505fe6e-8ba5-46b5-b16e-3b0caf0b713e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:11.014003 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b505fe6e-8ba5-46b5-b16e-3b0caf0b713e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:11.021816 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 165/322] 10.222.0.70 () {56 vars in 1376 bytes} [Thu Oct 27 09:24:10 2022] GET /identity/v3/role_assignments?group.id=de7aaa46b2d64a949a760bbb17bf2523&scope.project.id=78bd002b14ac4653ae0975911fcabf81&role.id=fa269b0e4dc54a24bb729398ce38231c => generated 611 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:11.026236 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b505fe6e-8ba5-46b5-b16e-3b0caf0b713e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:11.053785 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:11.066309 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:11.092366 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c74969-4188-4176-b416-aef2f45cce5b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:11.140426 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1454ce6e-bb8b-4041-8f87-49660784470b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:11.141699 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454ce6e-bb8b-4041-8f87-49660784470b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:11.141977 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454ce6e-bb8b-4041-8f87-49660784470b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:11.142249 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454ce6e-bb8b-4041-8f87-49660784470b None admin] PATH_INFO: `/v3/groups/de7aaa46b2d64a949a760bbb17bf2523` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:11.143711 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1454ce6e-bb8b-4041-8f87-49660784470b None admin] RBAC: Authorizing `identity:get_group(group_id=de7aaa46b2d64a949a760bbb17bf2523)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:11.145801 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1454ce6e-bb8b-4041-8f87-49660784470b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:11.148186 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 158/323] 10.222.0.70 () {56 vars in 1180 bytes} [Thu Oct 27 09:24:10 2022] GET /identity/v3/groups/de7aaa46b2d64a949a760bbb17bf2523 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:11.154768 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1454ce6e-bb8b-4041-8f87-49660784470b None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:11.155591 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b505fe6e-8ba5-46b5-b16e-3b0caf0b713e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:11.196145 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8448df-bb12-418c-afaa-582c12122e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:11.196966 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8448df-bb12-418c-afaa-582c12122e5e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:11.197316 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8448df-bb12-418c-afaa-582c12122e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:11.197684 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8448df-bb12-418c-afaa-582c12122e5e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:11.210383 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8448df-bb12-418c-afaa-582c12122e5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:24:11.217134 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-2b8448df-bb12-418c-afaa-582c12122e5e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 27 09:24:11.217134 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:11.217134 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:11.217134 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:11.217134 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:11.217134 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:11.217134 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:11.217134 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:11.217134 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:11.217134 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:11.217134 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:11.217134 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:11.217134 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:24:11.217134 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:24:11.217134 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:24:11.217134 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:24:11.218070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:24:11.218070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:24:11.218070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:24:11.218070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:24:11.218070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:11.218070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:11.218070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:11.218070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:11.218070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:11.218070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:11.218070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:11.218070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:11.218070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:11.218070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:11.218070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:11.218070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:11.219435 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:11.219435 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:11.219435 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:11.219435 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:11.219435 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:24:11.219435 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:24:11.219435 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:24:11.219435 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:24:11.219435 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:24:11.219435 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:24:11.219435 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 27 09:24:11.219435 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:11.219435 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 166/324] 10.222.0.70 () {56 vars in 1114 bytes} [Thu Oct 27 09:24:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:11.224380 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8448df-bb12-418c-afaa-582c12122e5e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:11.275898 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1454ce6e-bb8b-4041-8f87-49660784470b None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:11.316196 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:11.324824 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74554dd9-0078-48b9-8688-70432972ceb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:11.325441 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74554dd9-0078-48b9-8688-70432972ceb8 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:11.325723 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74554dd9-0078-48b9-8688-70432972ceb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:11.325996 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74554dd9-0078-48b9-8688-70432972ceb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:11.326748 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74554dd9-0078-48b9-8688-70432972ceb8 None admin] RBAC: Adding query filter params (group.id=de7aaa46b2d64a949a760bbb17bf2523, role.id=eea7cd23e01b4506a9eb6ab86f9adcd2, scope.project.id=e6e4e23ca6ee4fc0a033668401d7ba30) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:11.327027 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74554dd9-0078-48b9-8688-70432972ceb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:11.328471 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74554dd9-0078-48b9-8688-70432972ceb8 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:11.334932 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 159/325] 10.222.0.70 () {56 vars in 1376 bytes} [Thu Oct 27 09:24:11 2022] GET /identity/v3/role_assignments?group.id=de7aaa46b2d64a949a760bbb17bf2523&scope.project.id=e6e4e23ca6ee4fc0a033668401d7ba30&role.id=eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 611 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:11.335744 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74554dd9-0078-48b9-8688-70432972ceb8 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:11.346079 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8448df-bb12-418c-afaa-582c12122e5e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:11.355009 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:11.398711 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3858769e-6f20-45b0-ab35-f49e68106dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:11.399341 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858769e-6f20-45b0-ab35-f49e68106dd3 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:11.399613 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858769e-6f20-45b0-ab35-f49e68106dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:11.399877 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858769e-6f20-45b0-ab35-f49e68106dd3 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:11.400655 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3858769e-6f20-45b0-ab35-f49e68106dd3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:11.400962 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3858769e-6f20-45b0-ab35-f49e68106dd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:11.402365 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3858769e-6f20-45b0-ab35-f49e68106dd3 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:11.412574 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 167/326] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Oct 27 09:24:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:24:11.418507 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3858769e-6f20-45b0-ab35-f49e68106dd3 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:11.479668 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74554dd9-0078-48b9-8688-70432972ceb8 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:11.522746 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0278756-bc16-4d4f-bfa0-81e7f8c5e6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:11.523389 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0278756-bc16-4d4f-bfa0-81e7f8c5e6a7 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:11.524740 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0278756-bc16-4d4f-bfa0-81e7f8c5e6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:11.525029 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0278756-bc16-4d4f-bfa0-81e7f8c5e6a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:11.538106 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0278756-bc16-4d4f-bfa0-81e7f8c5e6a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:24:11.546017 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-e0278756-bc16-4d4f-bfa0-81e7f8c5e6a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 09:24:11.546017 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:11.546017 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:11.546017 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:11.546017 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:11.546017 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:11.546017 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:11.546017 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:11.546017 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:11.546017 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:11.546017 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:11.546017 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:11.546017 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:24:11.546017 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:24:11.546017 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:24:11.546017 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:24:11.547094 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:24:11.547094 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:24:11.547094 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:24:11.547094 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:24:11.547094 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:11.547094 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:11.547094 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:11.547094 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:11.547094 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:11.547094 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:11.547094 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:11.547094 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:11.547094 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:11.547094 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:11.547094 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:11.547094 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:11.548010 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:11.548010 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:11.548010 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:11.548010 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:11.548010 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:24:11.548010 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:24:11.548010 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:24:11.548010 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:24:11.548010 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:24:11.548010 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:24:11.548010 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 09:24:11.548010 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:11.548010 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 160/327] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Oct 27 09:24:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:11.554481 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0278756-bc16-4d4f-bfa0-81e7f8c5e6a7 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:11.558635 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3858769e-6f20-45b0-ab35-f49e68106dd3 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:11.605603 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8ed8d-a7a2-4433-ad67-7f23347674ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:11.606218 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8ed8d-a7a2-4433-ad67-7f23347674ba None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:11.606501 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8ed8d-a7a2-4433-ad67-7f23347674ba None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:11.606825 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8ed8d-a7a2-4433-ad67-7f23347674ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:11.617028 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:11.650816 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:11.684144 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0278756-bc16-4d4f-bfa0-81e7f8c5e6a7 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:11.698112 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d8ed8d-a7a2-4433-ad67-7f23347674ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:11.698275 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d8ed8d-a7a2-4433-ad67-7f23347674ba None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:11.724379 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e0f016-074c-4880-a386-a4b403bf9db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:11.725241 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e0f016-074c-4880-a386-a4b403bf9db4 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:11.725678 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e0f016-074c-4880-a386-a4b403bf9db4 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:11.726072 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e0f016-074c-4880-a386-a4b403bf9db4 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:11.727381 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e0f016-074c-4880-a386-a4b403bf9db4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:11.727689 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e0f016-074c-4880-a386-a4b403bf9db4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:11.729153 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e0f016-074c-4880-a386-a4b403bf9db4 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:11.738250 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 161/328] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Oct 27 09:24:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:11.744793 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e0f016-074c-4880-a386-a4b403bf9db4 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:11.816260 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 168/329] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Oct 27 09:24:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 27 09:24:11.822081 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8ed8d-a7a2-4433-ad67-7f23347674ba None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:11.863453 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e0f016-074c-4880-a386-a4b403bf9db4 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:11.864199 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:11.900118 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6013d396-4902-4640-ad44-1597a2843051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:11.900757 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6013d396-4902-4640-ad44-1597a2843051 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:11.901039 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6013d396-4902-4640-ad44-1597a2843051 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:11.901306 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6013d396-4902-4640-ad44-1597a2843051 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:11.905288 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:11.937768 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8ed8d-a7a2-4433-ad67-7f23347674ba None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:11.978133 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0afb44-3a0d-4a06-ac36-7536b251f108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:11.978765 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0afb44-3a0d-4a06-ac36-7536b251f108 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:11.979042 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0afb44-3a0d-4a06-ac36-7536b251f108 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:11.979313 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0afb44-3a0d-4a06-ac36-7536b251f108 None admin] PATH_INFO: `/v3/projects/e6e4e23ca6ee4fc0a033668401d7ba30` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:11.980424 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0afb44-3a0d-4a06-ac36-7536b251f108 None admin] RBAC: Authorizing `identity:get_project(project_id=e6e4e23ca6ee4fc0a033668401d7ba30)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:11.981902 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0afb44-3a0d-4a06-ac36-7536b251f108 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:11.983525 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 169/330] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Oct 27 09:24:11 2022] GET /identity/v3/projects/e6e4e23ca6ee4fc0a033668401d7ba30 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:24:11.985570 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6013d396-4902-4640-ad44-1597a2843051 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:11.987025 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6013d396-4902-4640-ad44-1597a2843051 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:11.988939 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0afb44-3a0d-4a06-ac36-7536b251f108 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:12.085234 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0afb44-3a0d-4a06-ac36-7536b251f108 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:12.087903 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 162/331] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Oct 27 09:24:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 27 09:24:12.093251 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:12.093605 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6013d396-4902-4640-ad44-1597a2843051 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:12.121504 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da8dc5e-2d72-465a-ac60-71760e0c7f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:12.122125 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da8dc5e-2d72-465a-ac60-71760e0c7f0c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:12.122401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da8dc5e-2d72-465a-ac60-71760e0c7f0c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:12.122669 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da8dc5e-2d72-465a-ac60-71760e0c7f0c None admin] PATH_INFO: `/v3/groups/de7aaa46b2d64a949a760bbb17bf2523` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:12.123756 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da8dc5e-2d72-465a-ac60-71760e0c7f0c None admin] RBAC: Authorizing `identity:get_group(group_id=de7aaa46b2d64a949a760bbb17bf2523)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:12.125270 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da8dc5e-2d72-465a-ac60-71760e0c7f0c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:12.126816 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 170/332] 10.222.0.70 () {56 vars in 1180 bytes} [Thu Oct 27 09:24:11 2022] GET /identity/v3/groups/de7aaa46b2d64a949a760bbb17bf2523 => generated 226 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:24:12.132374 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:12.132557 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da8dc5e-2d72-465a-ac60-71760e0c7f0c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:12.191385 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6013d396-4902-4640-ad44-1597a2843051 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:12.229077 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce48e8-d46e-4404-a072-6255ee4a1fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:12.229531 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da8dc5e-2d72-465a-ac60-71760e0c7f0c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:12.229694 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce48e8-d46e-4404-a072-6255ee4a1fc0 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:12.229973 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce48e8-d46e-4404-a072-6255ee4a1fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:12.230243 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce48e8-d46e-4404-a072-6255ee4a1fc0 None admin] PATH_INFO: `/v3/projects/d2f8e01e4d9d43978ddeb5033ead57f5` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:12.231349 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cce48e8-d46e-4404-a072-6255ee4a1fc0 None admin] RBAC: Authorizing `identity:get_project(project_id=d2f8e01e4d9d43978ddeb5033ead57f5)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:12.232829 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cce48e8-d46e-4404-a072-6255ee4a1fc0 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:12.234442 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 163/333] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Oct 27 09:24:12 2022] GET /identity/v3/projects/d2f8e01e4d9d43978ddeb5033ead57f5 => generated 344 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:24:12.239835 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce48e8-d46e-4404-a072-6255ee4a1fc0 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:12.262797 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d83181b-5177-4a11-9fb2-ed46595c00fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:12.263393 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d83181b-5177-4a11-9fb2-ed46595c00fc None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:12.263671 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d83181b-5177-4a11-9fb2-ed46595c00fc None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:12.263942 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d83181b-5177-4a11-9fb2-ed46595c00fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:12.264678 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d83181b-5177-4a11-9fb2-ed46595c00fc None admin] RBAC: Adding query filter params (group.id=de7aaa46b2d64a949a760bbb17bf2523, role.id=fa269b0e4dc54a24bb729398ce38231c, scope.project.id=e6e4e23ca6ee4fc0a033668401d7ba30) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:12.264985 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d83181b-5177-4a11-9fb2-ed46595c00fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:12.266386 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d83181b-5177-4a11-9fb2-ed46595c00fc None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:12.272859 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 171/334] 10.222.0.70 () {56 vars in 1376 bytes} [Thu Oct 27 09:24:12 2022] GET /identity/v3/role_assignments?group.id=de7aaa46b2d64a949a760bbb17bf2523&scope.project.id=e6e4e23ca6ee4fc0a033668401d7ba30&role.id=fa269b0e4dc54a24bb729398ce38231c => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:12.343901 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce48e8-d46e-4404-a072-6255ee4a1fc0 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:12.346627 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:12.380705 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06690d1a-79ef-4548-9f82-de96d9978b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:12.381347 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06690d1a-79ef-4548-9f82-de96d9978b04 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:12.381623 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06690d1a-79ef-4548-9f82-de96d9978b04 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:12.381893 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06690d1a-79ef-4548-9f82-de96d9978b04 None admin] PATH_INFO: `/v3/groups/7fea783eff8f47ddb9386a51f3485c77` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:12.383043 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06690d1a-79ef-4548-9f82-de96d9978b04 None admin] RBAC: Authorizing `identity:get_group(group_id=7fea783eff8f47ddb9386a51f3485c77)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:12.384495 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06690d1a-79ef-4548-9f82-de96d9978b04 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:12.386153 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 164/335] 10.222.0.70 () {56 vars in 1180 bytes} [Thu Oct 27 09:24:12 2022] GET /identity/v3/groups/7fea783eff8f47ddb9386a51f3485c77 => generated 229 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:24:12.392201 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d83181b-5177-4a11-9fb2-ed46595c00fc None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:12.435408 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:12.498885 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d83181b-5177-4a11-9fb2-ed46595c00fc None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:12.531970 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d863fd-5dde-44b9-9163-1afe26046fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:12.532599 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d863fd-5dde-44b9-9163-1afe26046fa3 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:12.532886 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d863fd-5dde-44b9-9163-1afe26046fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:12.533161 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d863fd-5dde-44b9-9163-1afe26046fa3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:12.533864 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d863fd-5dde-44b9-9163-1afe26046fa3 None admin] RBAC: Adding query filter params (group.id=7fea783eff8f47ddb9386a51f3485c77, role.id=0dfd839b7a194aee96f82f4ebc1620a7, scope.project.id=d2f8e01e4d9d43978ddeb5033ead57f5) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:12.534168 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d863fd-5dde-44b9-9163-1afe26046fa3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:12.535584 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d863fd-5dde-44b9-9163-1afe26046fa3 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:12.541837 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 172/336] 10.222.0.70 () {56 vars in 1376 bytes} [Thu Oct 27 09:24:12 2022] GET /identity/v3/role_assignments?group.id=7fea783eff8f47ddb9386a51f3485c77&scope.project.id=d2f8e01e4d9d43978ddeb5033ead57f5&role.id=0dfd839b7a194aee96f82f4ebc1620a7 => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:15.048358 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a0001-658d-493e-927d-27e0b0ba3d22 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:15.049347 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a0001-658d-493e-927d-27e0b0ba3d22 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:15.049837 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a0001-658d-493e-927d-27e0b0ba3d22 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:15.051430 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 165/337] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:15.066203 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5ce3c-148a-4eb5-88de-510343b77945 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:15.067107 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5ce3c-148a-4eb5-88de-510343b77945 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:15.067584 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5ce3c-148a-4eb5-88de-510343b77945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:15.093491 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-c6e5ce3c-148a-4eb5-88de-510343b77945 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:15.095270 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c6e5ce3c-148a-4eb5-88de-510343b77945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:15.178689 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 173/338] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:24:15.397042 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca3aef2-e224-4665-8425-e89ec5b24495 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:15.397698 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca3aef2-e224-4665-8425-e89ec5b24495 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:15.398169 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca3aef2-e224-4665-8425-e89ec5b24495 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:15.400236 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 166/339] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 27 09:24:15.405712 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c14ad0-b9e3-4f8b-8dfb-e32f57bb073e None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:15.406237 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c14ad0-b9e3-4f8b-8dfb-e32f57bb073e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:15.406706 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c14ad0-b9e3-4f8b-8dfb-e32f57bb073e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:15.433531 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-01c14ad0-b9e3-4f8b-8dfb-e32f57bb073e None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:15.435057 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-01c14ad0-b9e3-4f8b-8dfb-e32f57bb073e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:15.467720 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:15.524552 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 174/340] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:24:15.612758 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca3aef2-e224-4665-8425-e89ec5b24495 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:15.665018 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5647cbe-b00a-427e-b73a-7e5e77d20007 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:15.665196 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5647cbe-b00a-427e-b73a-7e5e77d20007 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:15.665473 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5647cbe-b00a-427e-b73a-7e5e77d20007 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:15.666793 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 175/341] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 27 09:24:15.673082 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba6b22-1d9d-4e64-bdfa-b5adef14072f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:15.673361 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba6b22-1d9d-4e64-bdfa-b5adef14072f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:15.673627 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba6b22-1d9d-4e64-bdfa-b5adef14072f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:15.699453 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-e1ba6b22-1d9d-4e64-bdfa-b5adef14072f None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:15.700677 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e1ba6b22-1d9d-4e64-bdfa-b5adef14072f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:15.795463 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca3aef2-e224-4665-8425-e89ec5b24495 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:15.827842 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 176/342] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 27 09:24:15.852458 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc71db1-a5ef-4613-9f74-d49250c999a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:15.853137 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc71db1-a5ef-4613-9f74-d49250c999a8 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:15.853413 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc71db1-a5ef-4613-9f74-d49250c999a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:15.853670 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc71db1-a5ef-4613-9f74-d49250c999a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:15.873627 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc71db1-a5ef-4613-9f74-d49250c999a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:15.875139 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc71db1-a5ef-4613-9f74-d49250c999a8 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:15.881435 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba6b22-1d9d-4e64-bdfa-b5adef14072f None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:15.898160 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-7fc71db1-a5ef-4613-9f74-d49250c999a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:15.898160 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:15.898160 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:15.898160 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:15.898160 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:15.898160 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:15.898160 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:15.898160 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:15.898160 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:15.898160 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:15.898160 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:15.898160 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:15.898160 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:24:15.898160 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:24:15.898160 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:24:15.898160 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:24:15.899188 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:15.899188 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:15.899188 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:15.899188 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:15.899188 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:15.899188 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:15.899188 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:15.899188 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:15.899188 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:15.899188 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:15.899188 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:15.899188 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:15.899188 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:15.899188 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:15.899188 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:15.899188 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:15.899188 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:24:15.900054 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:24:15.900054 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:15.900054 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:15.902426 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 167/343] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:24:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:15.904045 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4b437-1bda-4df2-9554-08d6b9d80ab7 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:15.904747 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4b437-1bda-4df2-9554-08d6b9d80ab7 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:15.905994 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4b437-1bda-4df2-9554-08d6b9d80ab7 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:15.906813 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 168/344] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:15.910992 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee4b437-1bda-4df2-9554-08d6b9d80ab7 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:15.936484 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:16.005042 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:16.016910 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee4b437-1bda-4df2-9554-08d6b9d80ab7 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:16.018951 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba6b22-1d9d-4e64-bdfa-b5adef14072f None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:16.051272 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e9df83-4a03-437c-a988-c22cbbfe44f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:16.051911 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e9df83-4a03-437c-a988-c22cbbfe44f9 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:16.052137 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e9df83-4a03-437c-a988-c22cbbfe44f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:16.052405 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e9df83-4a03-437c-a988-c22cbbfe44f9 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:16.053077 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e9df83-4a03-437c-a988-c22cbbfe44f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:16.053374 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e9df83-4a03-437c-a988-c22cbbfe44f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:16.055512 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e9df83-4a03-437c-a988-c22cbbfe44f9 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:16.064159 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2bbd57-1eab-4868-8e7d-e8a2d4a254d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:16.065171 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2bbd57-1eab-4868-8e7d-e8a2d4a254d1 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:16.065740 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2bbd57-1eab-4868-8e7d-e8a2d4a254d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:16.066240 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2bbd57-1eab-4868-8e7d-e8a2d4a254d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:16.068588 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 169/345] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:24:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:16.070011 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45d61f-b96f-40eb-87f3-5684a7db8303 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:16.071082 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45d61f-b96f-40eb-87f3-5684a7db8303 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:16.071566 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45d61f-b96f-40eb-87f3-5684a7db8303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:16.086282 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2bbd57-1eab-4868-8e7d-e8a2d4a254d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:16.088889 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2bbd57-1eab-4868-8e7d-e8a2d4a254d1 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:16.092395 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-6c45d61f-b96f-40eb-87f3-5684a7db8303 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:16.094656 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6c45d61f-b96f-40eb-87f3-5684a7db8303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:16.105740 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-0b2bbd57-1eab-4868-8e7d-e8a2d4a254d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:16.105740 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:16.105740 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:16.105740 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:16.105740 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:16.105740 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:16.105740 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:16.105740 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:16.105740 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:16.105740 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:16.105740 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:16.105740 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:16.105740 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:24:16.105740 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:24:16.105740 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:24:16.105740 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:24:16.106618 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:16.106618 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:16.106618 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:16.106618 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:16.106618 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:16.106618 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:16.106618 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:16.106618 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:16.106618 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:16.106618 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:16.106618 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:16.106618 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:16.106618 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:16.106618 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:16.106618 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:16.106618 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:16.106618 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:24:16.107469 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:24:16.107469 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:16.107469 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:16.110005 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 177/346] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:24:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:16.111013 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2bbd57-1eab-4868-8e7d-e8a2d4a254d1 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:16.172261 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 170/347] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:24:16.173391 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c45d61f-b96f-40eb-87f3-5684a7db8303 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:16.190461 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:16.230711 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2bbd57-1eab-4868-8e7d-e8a2d4a254d1 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:16.272679 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5160ea-049a-4a3b-abf5-066f55786a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:16.273321 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5160ea-049a-4a3b-abf5-066f55786a98 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:16.273596 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5160ea-049a-4a3b-abf5-066f55786a98 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:16.273861 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5160ea-049a-4a3b-abf5-066f55786a98 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:16.274786 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5160ea-049a-4a3b-abf5-066f55786a98 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:16.276320 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5160ea-049a-4a3b-abf5-066f55786a98 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:16.285912 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c45d61f-b96f-40eb-87f3-5684a7db8303 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:16.297168 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:16.303992 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 178/348] 10.222.0.70 () {60 vars in 1132 bytes} [Thu Oct 27 09:24:16 2022] POST /identity/v3/users => generated 246 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:24:16.315885 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5160ea-049a-4a3b-abf5-066f55786a98 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:16.339050 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36348ac-42bd-41aa-b648-5fe1d11d4785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:16.341650 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36348ac-42bd-41aa-b648-5fe1d11d4785 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:16.342279 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36348ac-42bd-41aa-b648-5fe1d11d4785 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:16.342954 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36348ac-42bd-41aa-b648-5fe1d11d4785 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:16.344612 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36348ac-42bd-41aa-b648-5fe1d11d4785 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:16.345636 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36348ac-42bd-41aa-b648-5fe1d11d4785 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:16.350429 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36348ac-42bd-41aa-b648-5fe1d11d4785 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:16.377876 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 171/349] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:24:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:16.387169 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36348ac-42bd-41aa-b648-5fe1d11d4785 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:16.466104 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5160ea-049a-4a3b-abf5-066f55786a98 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:16.513437 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:16.514427 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318eb68e-cdaf-464e-8a87-65ab879e8f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:16.515094 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318eb68e-cdaf-464e-8a87-65ab879e8f3e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:16.515444 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318eb68e-cdaf-464e-8a87-65ab879e8f3e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:16.515623 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318eb68e-cdaf-464e-8a87-65ab879e8f3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:16.531224 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318eb68e-cdaf-464e-8a87-65ab879e8f3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:16.532994 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318eb68e-cdaf-464e-8a87-65ab879e8f3e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:16.537688 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36348ac-42bd-41aa-b648-5fe1d11d4785 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:16.552125 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-318eb68e-cdaf-464e-8a87-65ab879e8f3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:16.552125 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:16.552125 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:16.552125 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:16.552125 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:16.552125 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:16.552125 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:16.552125 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:16.552125 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:16.552125 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:16.552125 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:16.552125 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:16.552125 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:24:16.552125 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:24:16.552125 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:24:16.552125 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:24:16.553148 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:16.553148 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:16.553148 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:16.553148 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:16.553148 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:16.553148 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:16.553148 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:16.553148 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:16.553148 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:16.553148 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:16.553148 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:16.553148 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:16.553148 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:16.553148 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:16.553148 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:16.553148 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:16.553148 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:24:16.554055 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:24:16.554055 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:16.554055 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:16.554055 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 179/350] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:24:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:16.554282 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318eb68e-cdaf-464e-8a87-65ab879e8f3e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:16.589657 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde5281b-336e-4558-8ede-d41cf1380978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:16.590547 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5281b-336e-4558-8ede-d41cf1380978 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:16.592283 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5281b-336e-4558-8ede-d41cf1380978 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:16.592283 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5281b-336e-4558-8ede-d41cf1380978 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:16.592761 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde5281b-336e-4558-8ede-d41cf1380978 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:16.594658 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde5281b-336e-4558-8ede-d41cf1380978 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:16.625931 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 172/351] 10.222.0.70 () {60 vars in 1132 bytes} [Thu Oct 27 09:24:16 2022] POST /identity/v3/users => generated 248 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:24:16.627032 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde5281b-336e-4558-8ede-d41cf1380978 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:16.629496 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:16.692243 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318eb68e-cdaf-464e-8a87-65ab879e8f3e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:16.736755 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdc697f-f3af-4e81-858e-a9b87b88329d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:16.739235 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc697f-f3af-4e81-858e-a9b87b88329d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:16.739840 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc697f-f3af-4e81-858e-a9b87b88329d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:16.740311 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc697f-f3af-4e81-858e-a9b87b88329d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:16.768272 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdc697f-f3af-4e81-858e-a9b87b88329d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:16.770170 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdc697f-f3af-4e81-858e-a9b87b88329d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:16.804212 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-fcdc697f-f3af-4e81-858e-a9b87b88329d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:16.804212 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:16.804212 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:16.804212 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:16.804212 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:16.804212 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:16.804212 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:16.804212 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:16.804212 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:16.804212 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:16.804212 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:16.804212 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:16.804212 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:24:16.804212 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:24:16.804212 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:24:16.804212 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:24:16.805248 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:16.805248 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:16.805248 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:16.805248 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:16.805248 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:16.805248 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:16.805248 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:16.805248 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:16.805248 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:16.805248 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:16.805248 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:16.805248 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:16.805248 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:16.805248 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:16.805248 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:16.805248 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:16.805248 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:24:16.806108 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:24:16.806108 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:16.806108 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:16.808348 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 180/352] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:24:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:16.818187 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdc697f-f3af-4e81-858e-a9b87b88329d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:16.852066 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde5281b-336e-4558-8ede-d41cf1380978 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:16.891490 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:16.900153 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c85592f-4ec2-4330-90b9-ef63d6ddd260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:16.900834 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c85592f-4ec2-4330-90b9-ef63d6ddd260 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:16.901122 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c85592f-4ec2-4330-90b9-ef63d6ddd260 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:16.901398 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c85592f-4ec2-4330-90b9-ef63d6ddd260 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:16.902050 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c85592f-4ec2-4330-90b9-ef63d6ddd260 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:16.902362 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c85592f-4ec2-4330-90b9-ef63d6ddd260 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:16.903768 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c85592f-4ec2-4330-90b9-ef63d6ddd260 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:16.919867 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 173/353] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:24:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:16.921405 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b2bcb-be5c-44c1-b8d9-4a499e68dd9e None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:16.921686 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b2bcb-be5c-44c1-b8d9-4a499e68dd9e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:16.921955 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b2bcb-be5c-44c1-b8d9-4a499e68dd9e None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:16.923834 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 174/354] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:16.928491 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b2bcb-be5c-44c1-b8d9-4a499e68dd9e None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:16.974426 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdc697f-f3af-4e81-858e-a9b87b88329d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:16.977733 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:17.009874 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c862a84-eb12-4bff-9368-aa3031a09c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:17.010400 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c862a84-eb12-4bff-9368-aa3031a09c0d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:17.010675 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c862a84-eb12-4bff-9368-aa3031a09c0d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:17.010935 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c862a84-eb12-4bff-9368-aa3031a09c0d None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:17.011573 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c862a84-eb12-4bff-9368-aa3031a09c0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:17.011957 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c862a84-eb12-4bff-9368-aa3031a09c0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:17.013379 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c862a84-eb12-4bff-9368-aa3031a09c0d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:17.025507 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 181/355] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:24:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:17.026749 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c949f92b-fc6a-406d-a23e-f5f554b12816 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:17.027523 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c949f92b-fc6a-406d-a23e-f5f554b12816 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:17.027921 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c949f92b-fc6a-406d-a23e-f5f554b12816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:17.046291 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-c949f92b-fc6a-406d-a23e-f5f554b12816 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:17.047619 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c949f92b-fc6a-406d-a23e-f5f554b12816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:17.052718 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b2bcb-be5c-44c1-b8d9-4a499e68dd9e None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:17.092151 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e7be51-c850-4d3c-8386-307a4f7b9b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:17.092943 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e7be51-c850-4d3c-8386-307a4f7b9b44 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:17.093710 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e7be51-c850-4d3c-8386-307a4f7b9b44 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:17.094090 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e7be51-c850-4d3c-8386-307a4f7b9b44 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:17.095156 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e7be51-c850-4d3c-8386-307a4f7b9b44 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:17.097399 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e7be51-c850-4d3c-8386-307a4f7b9b44 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:17.123746 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 182/356] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:24:17.125215 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c949f92b-fc6a-406d-a23e-f5f554b12816 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:17.132578 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 175/357] 10.222.0.70 () {60 vars in 1132 bytes} [Thu Oct 27 09:24:16 2022] POST /identity/v3/users => generated 248 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:24:17.287677 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:17.300807 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c949f92b-fc6a-406d-a23e-f5f554b12816 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:17.342434 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0675d34-6749-4541-8a10-23aa592b5269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:17.343065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0675d34-6749-4541-8a10-23aa592b5269 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:17.343351 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0675d34-6749-4541-8a10-23aa592b5269 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:17.343624 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0675d34-6749-4541-8a10-23aa592b5269 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:17.344272 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0675d34-6749-4541-8a10-23aa592b5269 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:17.346145 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0675d34-6749-4541-8a10-23aa592b5269 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:17.391292 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 183/358] 10.222.0.70 () {60 vars in 1132 bytes} [Thu Oct 27 09:24:17 2022] POST /identity/v3/users => generated 249 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:24:17.412288 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e7be51-c850-4d3c-8386-307a4f7b9b44 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:17.477092 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:17.539155 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e7be51-c850-4d3c-8386-307a4f7b9b44 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:17.578155 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b7a7d-1008-4522-97a1-9101f5f6da98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:17.578790 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b7a7d-1008-4522-97a1-9101f5f6da98 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:17.579068 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b7a7d-1008-4522-97a1-9101f5f6da98 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:17.579342 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b7a7d-1008-4522-97a1-9101f5f6da98 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:17.580117 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b7a7d-1008-4522-97a1-9101f5f6da98 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:17.581607 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b7a7d-1008-4522-97a1-9101f5f6da98 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:17.592682 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.sql.core [None req-4e4b7a7d-1008-4522-97a1-9101f5f6da98 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 27 09:24:17.592682 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 27 09:24:17.592682 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [parameters: {'id': '024e31fa03844e179abf33cc8816ef91', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 27 09:24:17.592682 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81437) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 27 09:24:17.610412 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-4e4b7a7d-1008-4522-97a1-9101f5f6da98 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 27 09:24:17.610412 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:17.610412 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 27 09:24:17.610412 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 27 09:24:17.610412 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 27 09:24:17.610412 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 27 09:24:17.610412 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 27 09:24:17.610412 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application result = self._query(query) Oct 27 09:24:17.610412 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 27 09:24:17.610412 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application conn.query(q) Oct 27 09:24:17.610412 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 27 09:24:17.610412 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 27 09:24:17.610412 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 27 09:24:17.610412 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application result.read() Oct 27 09:24:17.610412 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 27 09:24:17.611349 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 27 09:24:17.611349 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 27 09:24:17.611349 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 27 09:24:17.611349 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 27 09:24:17.611349 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 27 09:24:17.611349 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 27 09:24:17.611349 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 27 09:24:17.611349 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 27 09:24:17.611349 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:17.611349 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 27 09:24:17.611349 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:17.611349 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:17.611349 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 27 09:24:17.611349 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 27 09:24:17.611349 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 27 09:24:17.611349 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return ref.to_dict() Oct 27 09:24:17.611349 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 27 09:24:17.611349 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application next(self.gen) Oct 27 09:24:17.612289 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 27 09:24:17.612289 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application yield resource Oct 27 09:24:17.612289 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 27 09:24:17.612289 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application next(self.gen) Oct 27 09:24:17.612289 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 27 09:24:17.612289 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self.session.rollback() Oct 27 09:24:17.612289 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 27 09:24:17.612289 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self.force_reraise() Oct 27 09:24:17.612289 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 27 09:24:17.612289 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise self.value Oct 27 09:24:17.612289 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 27 09:24:17.612289 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 27 09:24:17.612289 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 27 09:24:17.612289 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application session.commit() Oct 27 09:24:17.612289 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 27 09:24:17.612289 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 27 09:24:17.612289 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 27 09:24:17.613839 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._prepare_impl() Oct 27 09:24:17.613839 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 27 09:24:17.613839 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self.session.flush() Oct 27 09:24:17.613839 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 27 09:24:17.613839 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._flush(objects) Oct 27 09:24:17.613839 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 27 09:24:17.613839 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 27 09:24:17.613839 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 27 09:24:17.613839 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application compat.raise_( Oct 27 09:24:17.613839 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 27 09:24:17.613839 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception Oct 27 09:24:17.613839 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 27 09:24:17.613839 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application flush_context.execute() Oct 27 09:24:17.613839 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 27 09:24:17.613839 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rec.execute(self) Oct 27 09:24:17.613839 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 27 09:24:17.613839 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 27 09:24:17.615913 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 27 09:24:17.615913 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 27 09:24:17.615913 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 27 09:24:17.615913 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 27 09:24:17.615913 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 27 09:24:17.615913 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 27 09:24:17.615913 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 27 09:24:17.615913 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 27 09:24:17.615913 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 27 09:24:17.615913 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 27 09:24:17.615913 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 27 09:24:17.615913 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 27 09:24:17.615913 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 27 09:24:17.615913 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 27 09:24:17.615913 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 27 09:24:17.615913 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception Oct 27 09:24:17.618467 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 27 09:24:17.618467 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 27 09:24:17.618467 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 27 09:24:17.618467 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 27 09:24:17.618467 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 27 09:24:17.618467 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application result = self._query(query) Oct 27 09:24:17.618467 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 27 09:24:17.618467 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application conn.query(q) Oct 27 09:24:17.618467 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 27 09:24:17.618467 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 27 09:24:17.618467 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 27 09:24:17.618467 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application result.read() Oct 27 09:24:17.618467 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 27 09:24:17.618467 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 27 09:24:17.618467 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 27 09:24:17.618467 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 27 09:24:17.620586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 27 09:24:17.620586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 27 09:24:17.620586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 27 09:24:17.620586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 27 09:24:17.620586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 27 09:24:17.620586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 27 09:24:17.620586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [parameters: {'id': '024e31fa03844e179abf33cc8816ef91', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 27 09:24:17.620586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 27 09:24:17.620586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:17.620586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 09:24:17.620586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:17.620586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:17.620586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:17.620586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:17.620586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:17.620586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:17.626119 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:17.626119 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:17.626119 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:17.626119 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:17.626119 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:17.626119 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:17.626119 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 27 09:24:17.626119 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 27 09:24:17.626119 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:17.626119 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:17.626119 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 27 09:24:17.626119 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 27 09:24:17.626119 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 27 09:24:17.626119 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 27 09:24:17.626119 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 27 09:24:17.626119 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:17.628073 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 176/359] 10.222.0.70 () {60 vars in 1131 bytes} [Thu Oct 27 09:24:17 2022] POST /identity/v3/roles => generated 144 bytes in 200 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:17.628325 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0675d34-6749-4541-8a10-23aa592b5269 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:17.735423 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0675d34-6749-4541-8a10-23aa592b5269 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:17.771668 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cbb813-5a88-46b8-bd35-42ee213c7a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:17.772592 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cbb813-5a88-46b8-bd35-42ee213c7a26 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:17.773069 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cbb813-5a88-46b8-bd35-42ee213c7a26 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:17.773532 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cbb813-5a88-46b8-bd35-42ee213c7a26 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:17.784706 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cbb813-5a88-46b8-bd35-42ee213c7a26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:24:17.790954 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-31cbb813-5a88-46b8-bd35-42ee213c7a26 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 27 09:24:17.790954 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:17.790954 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:17.790954 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:17.790954 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:17.790954 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:17.790954 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:17.790954 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:17.790954 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:17.790954 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:17.790954 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:17.790954 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:17.790954 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:24:17.790954 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:24:17.790954 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:24:17.790954 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:24:17.792032 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:24:17.792032 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:24:17.792032 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:24:17.792032 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:24:17.792032 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:17.792032 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:17.792032 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:17.792032 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:17.792032 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:17.792032 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:17.792032 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:17.792032 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:17.792032 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:17.792032 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:17.792032 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:17.792032 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:17.793175 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:17.793175 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:17.793175 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:17.793175 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:17.793175 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:24:17.793175 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:24:17.793175 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:24:17.793175 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:24:17.793175 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:24:17.793175 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:24:17.793175 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 27 09:24:17.793175 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:17.795189 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 184/360] 10.222.0.70 () {56 vars in 1114 bytes} [Thu Oct 27 09:24:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:17.801236 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b7a7d-1008-4522-97a1-9101f5f6da98 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:17.955910 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b7a7d-1008-4522-97a1-9101f5f6da98 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:17.970009 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:18.005353 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afda04c-4e76-4bab-a70a-b66ba7948bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:18.005967 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afda04c-4e76-4bab-a70a-b66ba7948bd7 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:18.006245 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afda04c-4e76-4bab-a70a-b66ba7948bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:18.006518 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afda04c-4e76-4bab-a70a-b66ba7948bd7 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:18.007183 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afda04c-4e76-4bab-a70a-b66ba7948bd7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:18.007484 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afda04c-4e76-4bab-a70a-b66ba7948bd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:18.008906 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afda04c-4e76-4bab-a70a-b66ba7948bd7 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:18.019516 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 177/361] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Oct 27 09:24:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:18.938295 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd740e-81c5-41e9-8ed8-743cd365b045 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:18.939314 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd740e-81c5-41e9-8ed8-743cd365b045 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:18.939791 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd740e-81c5-41e9-8ed8-743cd365b045 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:18.941602 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 185/362] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:18.947725 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221609a3-ca96-4d59-a622-21ccef84a2fe None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:18.948166 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221609a3-ca96-4d59-a622-21ccef84a2fe None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:18.948340 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221609a3-ca96-4d59-a622-21ccef84a2fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:18.974104 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-221609a3-ca96-4d59-a622-21ccef84a2fe None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:18.975368 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-221609a3-ca96-4d59-a622-21ccef84a2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:19.058799 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 178/363] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:24:19.373824 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafd740e-81c5-41e9-8ed8-743cd365b045 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:19.394464 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:19.446005 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febdd666-ab53-44d9-bd48-9f653f2a3be4 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:19.446446 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febdd666-ab53-44d9-bd48-9f653f2a3be4 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:19.446824 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febdd666-ab53-44d9-bd48-9f653f2a3be4 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:19.448292 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 179/364] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:19.454039 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7705fc-258b-4fac-b1a2-10be346312e0 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:19.454317 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7705fc-258b-4fac-b1a2-10be346312e0 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:19.454587 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7705fc-258b-4fac-b1a2-10be346312e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:19.481093 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-5a7705fc-258b-4fac-b1a2-10be346312e0 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:19.482649 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5a7705fc-258b-4fac-b1a2-10be346312e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:19.544374 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafd740e-81c5-41e9-8ed8-743cd365b045 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:19.582924 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 180/365] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 27 09:24:19.623544 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b509be6-42cd-4114-ac5e-050c8555b399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:19.624419 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b509be6-42cd-4114-ac5e-050c8555b399 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:19.624884 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b509be6-42cd-4114-ac5e-050c8555b399 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:19.625296 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b509be6-42cd-4114-ac5e-050c8555b399 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:19.642211 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b509be6-42cd-4114-ac5e-050c8555b399 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:24:19.647721 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-8b509be6-42cd-4114-ac5e-050c8555b399 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 27 09:24:19.647721 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:19.647721 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:19.647721 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:19.647721 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:19.647721 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:19.647721 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:19.647721 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:19.647721 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:19.647721 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:19.647721 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:19.647721 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:19.647721 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:24:19.647721 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:24:19.647721 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:24:19.647721 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:24:19.648845 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:24:19.648845 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:24:19.648845 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:24:19.648845 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:24:19.648845 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:19.648845 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:19.648845 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:19.648845 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:19.648845 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:19.648845 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:19.648845 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:19.648845 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:19.648845 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:19.648845 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:19.648845 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:19.648845 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:19.650709 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:19.650709 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:19.650709 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:19.650709 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:19.650709 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:24:19.650709 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:24:19.650709 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:24:19.650709 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:24:19.650709 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:24:19.650709 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:24:19.650709 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 27 09:24:19.650709 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:19.652672 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 186/366] 10.222.0.70 () {56 vars in 1102 bytes} [Thu Oct 27 09:24:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:19.658531 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7705fc-258b-4fac-b1a2-10be346312e0 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:19.712030 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:19.813591 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7705fc-258b-4fac-b1a2-10be346312e0 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:19.856297 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07549e0a-e3a1-4745-9849-70ece7f737d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:19.856959 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07549e0a-e3a1-4745-9849-70ece7f737d3 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:19.857306 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07549e0a-e3a1-4745-9849-70ece7f737d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:19.857582 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07549e0a-e3a1-4745-9849-70ece7f737d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:19.858261 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07549e0a-e3a1-4745-9849-70ece7f737d3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:19.858561 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07549e0a-e3a1-4745-9849-70ece7f737d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:19.859983 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07549e0a-e3a1-4745-9849-70ece7f737d3 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:19.870741 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 181/367] 10.222.0.70 () {56 vars in 1103 bytes} [Thu Oct 27 09:24:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:19.876287 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b509be6-42cd-4114-ac5e-050c8555b399 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:19.939902 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07549e0a-e3a1-4745-9849-70ece7f737d3 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:19.998114 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:20.033127 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b509be6-42cd-4114-ac5e-050c8555b399 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:20.071424 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:20.075668 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa674994-b0c1-45f3-8d5f-bf745f937e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:20.077054 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa674994-b0c1-45f3-8d5f-bf745f937e84 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:20.077540 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa674994-b0c1-45f3-8d5f-bf745f937e84 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:20.078010 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa674994-b0c1-45f3-8d5f-bf745f937e84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:20.081691 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07549e0a-e3a1-4745-9849-70ece7f737d3 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:20.091855 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa674994-b0c1-45f3-8d5f-bf745f937e84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:20.094263 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa674994-b0c1-45f3-8d5f-bf745f937e84 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:20.111968 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-fa674994-b0c1-45f3-8d5f-bf745f937e84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:20.111968 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:20.111968 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:20.111968 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:20.111968 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:20.111968 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:20.111968 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:20.111968 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:20.111968 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:20.111968 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:20.111968 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:20.111968 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:20.111968 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:24:20.111968 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:24:20.111968 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:24:20.111968 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:24:20.112991 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:20.112991 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:20.112991 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:20.112991 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:20.112991 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:20.112991 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:20.112991 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:20.112991 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:20.112991 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:20.112991 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:20.112991 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:20.112991 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:20.112991 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:20.112991 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:20.112991 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:20.112991 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:20.112991 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:24:20.113807 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:24:20.113807 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:20.113807 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:20.116161 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 187/368] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:24:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:20.118579 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bebfc9-6ece-40bb-a62d-59a3a6cdb81f None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:20.119029 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bebfc9-6ece-40bb-a62d-59a3a6cdb81f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:20.119445 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bebfc9-6ece-40bb-a62d-59a3a6cdb81f None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:20.121357 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 188/369] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:20.122408 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bebfc9-6ece-40bb-a62d-59a3a6cdb81f None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:20.126843 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dd42bd-4be7-4381-a04b-ecbfa16717fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:20.127924 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd42bd-4be7-4381-a04b-ecbfa16717fa None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:20.128470 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd42bd-4be7-4381-a04b-ecbfa16717fa None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:20.129070 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd42bd-4be7-4381-a04b-ecbfa16717fa None admin] PATH_INFO: `/v3/roles/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:20.141874 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dd42bd-4be7-4381-a04b-ecbfa16717fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:24:20.149371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-69dd42bd-4be7-4381-a04b-ecbfa16717fa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 27 09:24:20.149371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:20.149371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:20.149371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:20.149371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:20.149371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:20.149371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:20.149371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:20.149371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:20.149371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:20.149371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:20.149371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:20.149371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:24:20.149371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:24:20.149371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:24:20.149371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:24:20.150321 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:24:20.150321 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:24:20.150321 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:24:20.150321 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:24:20.150321 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:20.150321 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:20.150321 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:20.150321 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:20.150321 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:20.150321 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:20.150321 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:20.150321 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:20.150321 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:20.150321 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:20.150321 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:20.150321 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:20.151117 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:20.151117 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:20.151117 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:20.151117 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:20.151117 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:24:20.151117 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:24:20.151117 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:24:20.151117 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:24:20.151117 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:24:20.151117 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:24:20.151117 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 27 09:24:20.151117 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:20.153588 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 182/370] 10.222.0.70 () {56 vars in 1102 bytes} [Thu Oct 27 09:24:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 09:24:20.155072 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f69e1-c74b-48e3-8515-f24866eb30fe None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:20.155566 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f69e1-c74b-48e3-8515-f24866eb30fe None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:20.156031 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f69e1-c74b-48e3-8515-f24866eb30fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:20.175654 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-aa7f69e1-c74b-48e3-8515-f24866eb30fe None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:20.177205 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-aa7f69e1-c74b-48e3-8515-f24866eb30fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:20.247722 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bebfc9-6ece-40bb-a62d-59a3a6cdb81f None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:20.250775 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 183/371] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:24:20.257119 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7f69e1-c74b-48e3-8515-f24866eb30fe None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:20.278590 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:20.297582 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63631eb-ab38-4cbc-ba7b-f8d31108542a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:20.298271 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63631eb-ab38-4cbc-ba7b-f8d31108542a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:20.298612 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63631eb-ab38-4cbc-ba7b-f8d31108542a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:20.298951 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63631eb-ab38-4cbc-ba7b-f8d31108542a None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:20.299684 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63631eb-ab38-4cbc-ba7b-f8d31108542a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:20.300038 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63631eb-ab38-4cbc-ba7b-f8d31108542a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:20.301550 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63631eb-ab38-4cbc-ba7b-f8d31108542a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:20.314455 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 189/372] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:24:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:20.315521 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3246271-376a-49fd-bf7f-e44b5493e03a None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:20.315873 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3246271-376a-49fd-bf7f-e44b5493e03a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:20.316217 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3246271-376a-49fd-bf7f-e44b5493e03a None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:20.317608 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 190/373] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:20.322424 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3246271-376a-49fd-bf7f-e44b5493e03a None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:20.372885 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:20.395190 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7f69e1-c74b-48e3-8515-f24866eb30fe None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:20.432081 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3246271-376a-49fd-bf7f-e44b5493e03a None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:20.440570 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560546a8-bb46-4b33-b984-59fe29c32eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:20.441385 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560546a8-bb46-4b33-b984-59fe29c32eac None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:20.441813 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560546a8-bb46-4b33-b984-59fe29c32eac None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:20.442679 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560546a8-bb46-4b33-b984-59fe29c32eac None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:20.443490 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560546a8-bb46-4b33-b984-59fe29c32eac None admin] RBAC: Adding query filter params (name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:20.444351 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560546a8-bb46-4b33-b984-59fe29c32eac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:20.446411 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560546a8-bb46-4b33-b984-59fe29c32eac None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:20.458143 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 184/374] 10.222.0.70 () {56 vars in 1103 bytes} [Thu Oct 27 09:24:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:20.459629 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85236515-1627-46a0-86f9-7713fe574393 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:20.460039 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85236515-1627-46a0-86f9-7713fe574393 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:20.460460 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85236515-1627-46a0-86f9-7713fe574393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:20.473849 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a74b2da-4ca4-4bef-aed2-904437d31d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:20.474602 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a74b2da-4ca4-4bef-aed2-904437d31d84 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:20.474982 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a74b2da-4ca4-4bef-aed2-904437d31d84 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:20.475351 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a74b2da-4ca4-4bef-aed2-904437d31d84 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:20.483558 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a74b2da-4ca4-4bef-aed2-904437d31d84 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:20.484690 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-85236515-1627-46a0-86f9-7713fe574393 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:20.485267 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a74b2da-4ca4-4bef-aed2-904437d31d84 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:20.486180 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-85236515-1627-46a0-86f9-7713fe574393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:20.501570 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-6a74b2da-4ca4-4bef-aed2-904437d31d84 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 27 09:24:20.501570 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:20.501570 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:20.501570 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:20.501570 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:20.501570 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:20.501570 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:20.501570 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:20.501570 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:20.501570 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:20.501570 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:20.501570 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:20.501570 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 09:24:20.501570 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 09:24:20.501570 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 09:24:20.501570 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 09:24:20.502541 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:20.502541 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:20.502541 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 09:24:20.502541 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:24:20.502541 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 09:24:20.502541 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:24:20.502541 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:20.502541 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:20.502541 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:20.502541 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:20.502541 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:20.502541 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:20.502541 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:20.502541 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:20.502541 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:20.502541 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:20.502541 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:20.502541 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:20.503425 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:20.503425 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:20.503425 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 09:24:20.503425 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 09:24:20.503425 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 09:24:20.503425 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 09:24:20.503425 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 09:24:20.503425 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 09:24:20.503425 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 27 09:24:20.503425 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:20.503425 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 191/375] 10.222.0.70 () {56 vars in 1093 bytes} [Thu Oct 27 09:24:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:20.503988 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a74b2da-4ca4-4bef-aed2-904437d31d84 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:20.554536 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:20.577476 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 185/376] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:24:20.578636 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85236515-1627-46a0-86f9-7713fe574393 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:20.634909 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a74b2da-4ca4-4bef-aed2-904437d31d84 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:20.673703 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582c28aa-eaa8-48e5-8639-ff5f3ecf6d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:20.674336 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c28aa-eaa8-48e5-8639-ff5f3ecf6d07 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:20.674614 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c28aa-eaa8-48e5-8639-ff5f3ecf6d07 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:20.674885 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c28aa-eaa8-48e5-8639-ff5f3ecf6d07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:20.674987 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:20.688053 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582c28aa-eaa8-48e5-8639-ff5f3ecf6d07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:20.689587 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582c28aa-eaa8-48e5-8639-ff5f3ecf6d07 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:20.704223 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85236515-1627-46a0-86f9-7713fe574393 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:20.706649 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-582c28aa-eaa8-48e5-8639-ff5f3ecf6d07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:20.706649 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:20.706649 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:20.706649 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:20.706649 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:20.706649 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:20.706649 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:20.706649 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:20.706649 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:20.706649 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:20.706649 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:20.706649 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:20.706649 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:24:20.706649 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:24:20.706649 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:24:20.706649 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:24:20.707517 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:20.707517 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:20.707517 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:20.707517 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:20.707517 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:20.707517 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:20.707517 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:20.707517 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:20.707517 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:20.707517 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:20.707517 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:20.707517 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:20.707517 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:20.707517 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:20.707517 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:20.707517 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:20.707517 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:24:20.708348 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:24:20.708348 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:20.708348 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:20.712852 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 192/377] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:24:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:20.713684 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582c28aa-eaa8-48e5-8639-ff5f3ecf6d07 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:20.747415 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1187630-8b8b-42a7-abab-99ac40acffdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:20.748268 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1187630-8b8b-42a7-abab-99ac40acffdd None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:20.748560 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1187630-8b8b-42a7-abab-99ac40acffdd None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:20.749235 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1187630-8b8b-42a7-abab-99ac40acffdd None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:20.750241 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1187630-8b8b-42a7-abab-99ac40acffdd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:20.750540 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1187630-8b8b-42a7-abab-99ac40acffdd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:20.752243 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1187630-8b8b-42a7-abab-99ac40acffdd None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:20.794208 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 186/378] 10.222.0.70 () {56 vars in 1133 bytes} [Thu Oct 27 09:24:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:20.795157 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1187630-8b8b-42a7-abab-99ac40acffdd None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:20.828285 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:20.837574 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582c28aa-eaa8-48e5-8639-ff5f3ecf6d07 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:20.877805 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6654914-4101-46a2-8449-6265a0dfbd5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:20.878435 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6654914-4101-46a2-8449-6265a0dfbd5e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:20.878709 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6654914-4101-46a2-8449-6265a0dfbd5e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:20.878975 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6654914-4101-46a2-8449-6265a0dfbd5e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:20.889764 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6654914-4101-46a2-8449-6265a0dfbd5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:24:20.895168 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-e6654914-4101-46a2-8449-6265a0dfbd5e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 27 09:24:20.895168 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:20.895168 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:20.895168 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:20.895168 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:20.895168 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:20.895168 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:20.895168 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:20.895168 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:20.895168 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:20.895168 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:20.895168 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:20.895168 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:24:20.895168 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:24:20.895168 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:24:20.895168 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:24:20.896113 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:24:20.896113 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:24:20.896113 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:24:20.896113 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:24:20.896113 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:20.896113 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:20.896113 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:20.896113 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:20.896113 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:20.896113 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:20.896113 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:20.896113 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:20.896113 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:20.896113 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:20.896113 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:20.896113 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:20.896941 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:20.896941 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:20.896941 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:20.896941 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:20.896941 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:24:20.896941 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:24:20.896941 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:24:20.896941 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:24:20.896941 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:24:20.896941 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:24:20.896941 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 27 09:24:20.896941 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:20.896941 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 193/379] 10.222.0.70 () {56 vars in 1102 bytes} [Thu Oct 27 09:24:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:20.898873 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6654914-4101-46a2-8449-6265a0dfbd5e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:20.909128 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1187630-8b8b-42a7-abab-99ac40acffdd None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:20.949316 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8798da74-e623-49de-83d5-0c9a22d10e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:20.949947 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8798da74-e623-49de-83d5-0c9a22d10e19 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:20.950228 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8798da74-e623-49de-83d5-0c9a22d10e19 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:20.950497 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8798da74-e623-49de-83d5-0c9a22d10e19 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:20.951524 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8798da74-e623-49de-83d5-0c9a22d10e19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:20.951829 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8798da74-e623-49de-83d5-0c9a22d10e19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:20.953671 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8798da74-e623-49de-83d5-0c9a22d10e19 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:20.955860 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:20.968913 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 187/380] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:24:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:20.969972 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8798da74-e623-49de-83d5-0c9a22d10e19 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:21.019303 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6654914-4101-46a2-8449-6265a0dfbd5e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:21.054562 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44291b6-4a80-484a-9477-6f0dc340dd30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:21.055182 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44291b6-4a80-484a-9477-6f0dc340dd30 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:21.055460 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44291b6-4a80-484a-9477-6f0dc340dd30 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:21.055728 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44291b6-4a80-484a-9477-6f0dc340dd30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:21.066251 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44291b6-4a80-484a-9477-6f0dc340dd30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:21.067669 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44291b6-4a80-484a-9477-6f0dc340dd30 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:21.073498 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8798da74-e623-49de-83d5-0c9a22d10e19 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:21.082434 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-d44291b6-4a80-484a-9477-6f0dc340dd30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:21.082434 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:21.082434 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:21.082434 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:21.082434 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:21.082434 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:21.082434 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:21.082434 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:21.082434 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:21.082434 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:21.082434 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:21.082434 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:21.082434 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:24:21.082434 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:24:21.082434 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:24:21.082434 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:24:21.083611 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:21.083611 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:21.083611 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:21.083611 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:21.083611 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:21.083611 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:21.083611 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:21.083611 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:21.083611 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:21.083611 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:21.083611 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:21.083611 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:21.083611 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:21.083611 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:21.083611 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:21.083611 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:21.083611 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:24:21.084905 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:24:21.084905 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:21.084905 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:21.084905 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 194/381] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:24:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:21.084905 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44291b6-4a80-484a-9477-6f0dc340dd30 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:21.093529 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:21.112307 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7698be-1591-48cd-8d98-51d5b18525c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:21.112955 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7698be-1591-48cd-8d98-51d5b18525c1 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:21.113230 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7698be-1591-48cd-8d98-51d5b18525c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:21.113494 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7698be-1591-48cd-8d98-51d5b18525c1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:21.124249 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7698be-1591-48cd-8d98-51d5b18525c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:24:21.129600 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-9d7698be-1591-48cd-8d98-51d5b18525c1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 27 09:24:21.129600 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:21.129600 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:21.129600 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:21.129600 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:21.129600 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:21.129600 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:21.129600 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:21.129600 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:21.129600 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:21.129600 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:21.129600 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:21.129600 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:24:21.129600 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:24:21.129600 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:24:21.129600 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:24:21.130535 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:24:21.130535 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:24:21.130535 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:24:21.130535 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:24:21.130535 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:21.130535 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:21.130535 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:21.130535 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:21.130535 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:21.130535 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:21.130535 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:21.130535 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:21.130535 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:21.130535 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:21.130535 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:21.130535 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:21.131312 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:21.131312 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:21.131312 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:21.131312 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:21.131312 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:24:21.131312 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:24:21.131312 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:24:21.131312 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:24:21.131312 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:24:21.131312 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:24:21.131312 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 27 09:24:21.131312 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:21.131312 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 188/382] 10.222.0.70 () {56 vars in 1102 bytes} [Thu Oct 27 09:24:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:21.133529 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7698be-1591-48cd-8d98-51d5b18525c1 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:21.183993 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44291b6-4a80-484a-9477-6f0dc340dd30 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:21.187283 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:21.217555 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9d02aa-6946-4b33-8495-0505b4f5ec28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:21.218153 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d02aa-6946-4b33-8495-0505b4f5ec28 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:21.218433 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d02aa-6946-4b33-8495-0505b4f5ec28 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:21.218702 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d02aa-6946-4b33-8495-0505b4f5ec28 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:21.219358 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9d02aa-6946-4b33-8495-0505b4f5ec28 None admin] RBAC: Adding query filter params (name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:21.219913 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9d02aa-6946-4b33-8495-0505b4f5ec28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:21.221309 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9d02aa-6946-4b33-8495-0505b4f5ec28 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:21.229595 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 195/383] 10.222.0.70 () {56 vars in 1103 bytes} [Thu Oct 27 09:24:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:21.230735 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54404c55-664a-4ae8-bc0c-9f995caaf3e6 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:21.230967 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7698be-1591-48cd-8d98-51d5b18525c1 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:21.231078 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54404c55-664a-4ae8-bc0c-9f995caaf3e6 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:21.231275 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54404c55-664a-4ae8-bc0c-9f995caaf3e6 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:21.232460 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 196/384] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:21.233170 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54404c55-664a-4ae8-bc0c-9f995caaf3e6 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:21.267652 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b1724e-9721-4e01-b5f1-60c5483a3bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:21.268289 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1724e-9721-4e01-b5f1-60c5483a3bf0 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:21.268588 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1724e-9721-4e01-b5f1-60c5483a3bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:21.268862 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1724e-9721-4e01-b5f1-60c5483a3bf0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:21.275964 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b1724e-9721-4e01-b5f1-60c5483a3bf0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:21.277447 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b1724e-9721-4e01-b5f1-60c5483a3bf0 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:21.292178 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-f9b1724e-9721-4e01-b5f1-60c5483a3bf0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 27 09:24:21.292178 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:21.292178 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:21.292178 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:21.292178 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:21.292178 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:21.292178 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:21.292178 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:21.292178 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:21.292178 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:21.292178 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:21.292178 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:21.292178 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 09:24:21.292178 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 09:24:21.292178 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 09:24:21.292178 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 09:24:21.293141 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:21.293141 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:21.293141 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 09:24:21.293141 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:24:21.293141 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 09:24:21.293141 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:24:21.293141 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:21.293141 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:21.293141 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:21.293141 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:21.293141 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:21.293141 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:21.293141 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:21.293141 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:21.293141 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:21.293141 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:21.293141 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:21.293141 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:21.294058 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:21.294058 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:21.294058 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 09:24:21.294058 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 09:24:21.294058 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 09:24:21.294058 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 09:24:21.294058 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 09:24:21.294058 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 09:24:21.294058 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 27 09:24:21.294058 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:21.294058 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 189/385] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Oct 27 09:24:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:21.294058 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b1724e-9721-4e01-b5f1-60c5483a3bf0 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:21.307744 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:21.343282 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54404c55-664a-4ae8-bc0c-9f995caaf3e6 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:21.376071 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d172d336-4de3-4d85-a784-a643ac2c649c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:21.376697 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d172d336-4de3-4d85-a784-a643ac2c649c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:21.376978 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d172d336-4de3-4d85-a784-a643ac2c649c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:21.377248 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d172d336-4de3-4d85-a784-a643ac2c649c None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:21.377878 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d172d336-4de3-4d85-a784-a643ac2c649c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:21.378180 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d172d336-4de3-4d85-a784-a643ac2c649c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:21.379539 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d172d336-4de3-4d85-a784-a643ac2c649c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:21.391554 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 197/386] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:24:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:21.392335 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d172d336-4de3-4d85-a784-a643ac2c649c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:21.393888 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b1724e-9721-4e01-b5f1-60c5483a3bf0 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:21.408233 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:21.428906 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88303fce-9448-4944-a1d5-aa7db28bcae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:21.430063 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88303fce-9448-4944-a1d5-aa7db28bcae9 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:21.430341 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88303fce-9448-4944-a1d5-aa7db28bcae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:21.430609 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88303fce-9448-4944-a1d5-aa7db28bcae9 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:21.431255 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88303fce-9448-4944-a1d5-aa7db28bcae9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:21.431556 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88303fce-9448-4944-a1d5-aa7db28bcae9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:21.432981 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88303fce-9448-4944-a1d5-aa7db28bcae9 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:21.441178 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 190/387] 10.222.0.70 () {56 vars in 1103 bytes} [Thu Oct 27 09:24:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:21.442286 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d8d6cb-75ed-457d-8a6f-ce90167eb091 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:21.442569 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d8d6cb-75ed-457d-8a6f-ce90167eb091 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:21.442845 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d8d6cb-75ed-457d-8a6f-ce90167eb091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:21.460723 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-30d8d6cb-75ed-457d-8a6f-ce90167eb091 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:21.461902 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-30d8d6cb-75ed-457d-8a6f-ce90167eb091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:21.495573 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d172d336-4de3-4d85-a784-a643ac2c649c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:21.520964 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:21.528548 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 191/388] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:24:21.529074 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0272cd5-173d-414d-9551-d9ab5ed02ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:21.529551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d8d6cb-75ed-457d-8a6f-ce90167eb091 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:21.529721 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0272cd5-173d-414d-9551-d9ab5ed02ca6 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:21.529977 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0272cd5-173d-414d-9551-d9ab5ed02ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:21.530230 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0272cd5-173d-414d-9551-d9ab5ed02ca6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:21.543243 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0272cd5-173d-414d-9551-d9ab5ed02ca6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:21.544844 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0272cd5-173d-414d-9551-d9ab5ed02ca6 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:21.564294 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-e0272cd5-173d-414d-9551-d9ab5ed02ca6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:21.564294 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:21.564294 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:21.564294 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:21.564294 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:21.564294 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:21.564294 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:21.564294 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:21.564294 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:21.564294 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:21.564294 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:21.564294 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:21.564294 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:24:21.564294 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:24:21.564294 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:24:21.564294 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:24:21.565379 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:21.565379 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:21.565379 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:21.565379 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:21.565379 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:21.565379 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:21.565379 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:21.565379 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:21.565379 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:21.565379 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:21.565379 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:21.565379 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:21.565379 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:21.565379 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:21.565379 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:21.565379 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:21.565379 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:24:21.566213 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:24:21.566213 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:21.566213 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:21.566372 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 198/389] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:24:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:21.571384 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0272cd5-173d-414d-9551-d9ab5ed02ca6 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:21.635515 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d8d6cb-75ed-457d-8a6f-ce90167eb091 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:21.642783 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:21.671066 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fbe44b-02b5-44b9-b570-2c0e10a0f23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:21.672191 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fbe44b-02b5-44b9-b570-2c0e10a0f23c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:21.672683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fbe44b-02b5-44b9-b570-2c0e10a0f23c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:21.672961 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fbe44b-02b5-44b9-b570-2c0e10a0f23c None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:21.673680 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fbe44b-02b5-44b9-b570-2c0e10a0f23c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:21.673984 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fbe44b-02b5-44b9-b570-2c0e10a0f23c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:21.675413 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fbe44b-02b5-44b9-b570-2c0e10a0f23c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:21.677105 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0272cd5-173d-414d-9551-d9ab5ed02ca6 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:21.697679 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 192/390] 10.222.0.70 () {56 vars in 1137 bytes} [Thu Oct 27 09:24:21 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:21.697679 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fbe44b-02b5-44b9-b570-2c0e10a0f23c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:21.717959 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0742a-d90f-45ed-bbba-0d58fda86e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:21.718580 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0742a-d90f-45ed-bbba-0d58fda86e70 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:21.718857 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0742a-d90f-45ed-bbba-0d58fda86e70 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:21.719212 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0742a-d90f-45ed-bbba-0d58fda86e70 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:21.727090 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb0742a-d90f-45ed-bbba-0d58fda86e70 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:21.728700 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb0742a-d90f-45ed-bbba-0d58fda86e70 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:21.741361 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-6fb0742a-d90f-45ed-bbba-0d58fda86e70 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:24:21.741361 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:21.741361 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:21.741361 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:21.741361 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:21.741361 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:21.741361 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:21.741361 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:21.741361 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:21.741361 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:21.741361 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:21.741361 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:21.741361 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 09:24:21.741361 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 09:24:21.741361 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 09:24:21.742791 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 09:24:21.742791 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:21.742791 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:21.742791 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:21.742791 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:21.742791 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:21.742791 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:21.742791 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:21.742791 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:21.742791 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:21.742791 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:21.742791 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:21.742791 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:21.742791 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:21.742791 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:21.742791 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:21.742791 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:21.743680 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 09:24:21.743680 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 09:24:21.743680 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 09:24:21.743680 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 09:24:21.743680 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 09:24:21.743680 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 09:24:21.743680 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:24:21.743680 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:21.743680 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 199/391] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Oct 27 09:24:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:21.744147 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0742a-d90f-45ed-bbba-0d58fda86e70 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:21.789452 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:21.800821 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fbe44b-02b5-44b9-b570-2c0e10a0f23c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:21.834480 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b57f3b-c5c0-4ab2-b0ba-c462fdda6b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:21.835099 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b57f3b-c5c0-4ab2-b0ba-c462fdda6b02 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:21.835373 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b57f3b-c5c0-4ab2-b0ba-c462fdda6b02 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:21.836171 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b57f3b-c5c0-4ab2-b0ba-c462fdda6b02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:21.846120 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b57f3b-c5c0-4ab2-b0ba-c462fdda6b02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:21.847557 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b57f3b-c5c0-4ab2-b0ba-c462fdda6b02 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:21.850437 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0742a-d90f-45ed-bbba-0d58fda86e70 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:21.862288 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-f6b57f3b-c5c0-4ab2-b0ba-c462fdda6b02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:21.862288 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:21.862288 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:21.862288 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:21.862288 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:21.862288 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:21.862288 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:21.862288 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:21.862288 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:21.862288 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:21.862288 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:21.862288 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:21.862288 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:24:21.862288 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:24:21.862288 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:24:21.862288 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:24:21.863148 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:21.863148 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:21.863148 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:21.863148 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:21.863148 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:21.863148 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:21.863148 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:21.863148 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:21.863148 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:21.863148 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:21.863148 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:21.863148 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:21.863148 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:21.863148 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:21.863148 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:21.863148 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:21.863148 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:24:21.863975 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:24:21.863975 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:21.863975 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:21.863975 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 193/392] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:24:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:21.864195 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b57f3b-c5c0-4ab2-b0ba-c462fdda6b02 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:21.881598 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:21.891411 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f267b-91e3-4dfa-aac3-01c6ecfb50d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:21.891953 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f267b-91e3-4dfa-aac3-01c6ecfb50d7 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:21.892232 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f267b-91e3-4dfa-aac3-01c6ecfb50d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:21.892503 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f267b-91e3-4dfa-aac3-01c6ecfb50d7 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:21.893258 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7f267b-91e3-4dfa-aac3-01c6ecfb50d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:21.893568 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7f267b-91e3-4dfa-aac3-01c6ecfb50d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:21.895118 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7f267b-91e3-4dfa-aac3-01c6ecfb50d7 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:21.907430 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 200/393] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:24:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:21.908218 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f267b-91e3-4dfa-aac3-01c6ecfb50d7 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:21.964719 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b57f3b-c5c0-4ab2-b0ba-c462fdda6b02 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:21.998836 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aaead2-5eb3-4c5d-9629-cb729004a851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:21.999464 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aaead2-5eb3-4c5d-9629-cb729004a851 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:21.999740 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aaead2-5eb3-4c5d-9629-cb729004a851 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:22.000010 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aaead2-5eb3-4c5d-9629-cb729004a851 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:22.002162 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:22.005788 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f267b-91e3-4dfa-aac3-01c6ecfb50d7 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:22.010091 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aaead2-5eb3-4c5d-9629-cb729004a851 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:22.011572 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aaead2-5eb3-4c5d-9629-cb729004a851 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:22.026465 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-33aaead2-5eb3-4c5d-9629-cb729004a851 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:22.026465 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:22.026465 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:22.026465 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:22.026465 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:22.026465 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:22.026465 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:22.026465 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:22.026465 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:22.026465 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:22.026465 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:22.026465 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:22.026465 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:24:22.026465 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:24:22.026465 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:24:22.026465 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:24:22.027501 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:22.027501 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:22.027501 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:22.027501 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:22.027501 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:22.027501 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:22.027501 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:22.027501 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:22.027501 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:22.027501 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:22.027501 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:22.027501 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:22.027501 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:22.027501 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:22.027501 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:22.027501 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:22.027501 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:24:22.028328 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:24:22.028328 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:22.028328 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:22.028328 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 194/394] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:24:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:22.028328 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aaead2-5eb3-4c5d-9629-cb729004a851 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:22.043745 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442cbec5-c204-4c41-a06f-53b240925a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:22.044351 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442cbec5-c204-4c41-a06f-53b240925a30 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:22.044661 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442cbec5-c204-4c41-a06f-53b240925a30 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:22.044933 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442cbec5-c204-4c41-a06f-53b240925a30 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:22.045631 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442cbec5-c204-4c41-a06f-53b240925a30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:22.045915 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442cbec5-c204-4c41-a06f-53b240925a30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:22.047349 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442cbec5-c204-4c41-a06f-53b240925a30 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:22.064531 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 201/395] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Oct 27 09:24:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:22.065327 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442cbec5-c204-4c41-a06f-53b240925a30 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:22.094978 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:22.130186 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aaead2-5eb3-4c5d-9629-cb729004a851 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:22.163303 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362a8b2d-cc30-40da-94a9-ed6de145d3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:22.164413 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a8b2d-cc30-40da-94a9-ed6de145d3b6 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:22.164716 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a8b2d-cc30-40da-94a9-ed6de145d3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:22.164981 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a8b2d-cc30-40da-94a9-ed6de145d3b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:22.165172 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442cbec5-c204-4c41-a06f-53b240925a30 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:22.175073 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362a8b2d-cc30-40da-94a9-ed6de145d3b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:22.176510 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362a8b2d-cc30-40da-94a9-ed6de145d3b6 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:22.190388 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-362a8b2d-cc30-40da-94a9-ed6de145d3b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:22.190388 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:22.190388 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:22.190388 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:22.190388 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:22.190388 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:22.190388 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:22.190388 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:22.190388 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:22.190388 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:22.190388 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:22.190388 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:22.190388 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:24:22.190388 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:24:22.190388 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:24:22.190388 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:24:22.191368 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:22.191368 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:22.191368 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:22.191368 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:22.191368 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:22.191368 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:22.191368 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:22.191368 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:22.191368 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:22.191368 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:22.191368 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:22.191368 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:22.191368 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:22.191368 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:22.191368 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:22.191368 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:22.191368 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:24:22.192221 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:24:22.192221 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:22.192221 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:22.192221 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 195/396] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:24:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:22.192442 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362a8b2d-cc30-40da-94a9-ed6de145d3b6 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:22.202085 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd04508a-5928-4fdf-90e9-2daffd9e9915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:22.202685 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd04508a-5928-4fdf-90e9-2daffd9e9915 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:22.202959 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd04508a-5928-4fdf-90e9-2daffd9e9915 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:22.203222 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd04508a-5928-4fdf-90e9-2daffd9e9915 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:22.203848 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd04508a-5928-4fdf-90e9-2daffd9e9915 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:22.204145 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd04508a-5928-4fdf-90e9-2daffd9e9915 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:22.205524 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd04508a-5928-4fdf-90e9-2daffd9e9915 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:22.218014 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 202/397] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:24:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:22.218800 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd04508a-5928-4fdf-90e9-2daffd9e9915 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:22.242466 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:22.297425 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362a8b2d-cc30-40da-94a9-ed6de145d3b6 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:22.316512 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:22.322222 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd04508a-5928-4fdf-90e9-2daffd9e9915 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:22.330812 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9d7763-7ad0-4f97-bfac-a37f13ac9359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:22.331430 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d7763-7ad0-4f97-bfac-a37f13ac9359 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:22.331708 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d7763-7ad0-4f97-bfac-a37f13ac9359 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:22.331979 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d7763-7ad0-4f97-bfac-a37f13ac9359 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:22.339814 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9d7763-7ad0-4f97-bfac-a37f13ac9359 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:22.341294 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9d7763-7ad0-4f97-bfac-a37f13ac9359 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:22.353125 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-bc9d7763-7ad0-4f97-bfac-a37f13ac9359 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 27 09:24:22.353125 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:22.353125 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:22.353125 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:22.353125 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:22.353125 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:22.353125 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:22.353125 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:22.353125 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:22.353125 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:22.353125 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:22.353125 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:22.353125 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 09:24:22.353125 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 09:24:22.353125 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 09:24:22.353125 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 09:24:22.354150 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:22.354150 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:22.354150 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 09:24:22.354150 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:24:22.354150 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 09:24:22.354150 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:24:22.354150 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:22.354150 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:22.354150 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:22.354150 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:22.354150 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:22.354150 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:22.354150 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:22.354150 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:22.354150 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:22.354150 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:22.354150 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:22.354150 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:22.355043 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:22.355043 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:22.355043 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 09:24:22.355043 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 09:24:22.355043 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 09:24:22.355043 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 09:24:22.355043 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 09:24:22.355043 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 09:24:22.355043 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 27 09:24:22.355043 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:22.355043 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 196/398] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Oct 27 09:24:22 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:22.355598 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9d7763-7ad0-4f97-bfac-a37f13ac9359 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:22.361708 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4b565c-6f91-4a2d-8d89-4c06b93df7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:22.362306 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4b565c-6f91-4a2d-8d89-4c06b93df7af None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:22.362581 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4b565c-6f91-4a2d-8d89-4c06b93df7af None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:22.362848 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4b565c-6f91-4a2d-8d89-4c06b93df7af None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:22.363475 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4b565c-6f91-4a2d-8d89-4c06b93df7af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:22.363779 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4b565c-6f91-4a2d-8d89-4c06b93df7af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:22.365160 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4b565c-6f91-4a2d-8d89-4c06b93df7af None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:22.377446 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 203/399] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:24:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:22.378216 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4b565c-6f91-4a2d-8d89-4c06b93df7af None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:22.455449 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:22.455614 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9d7763-7ad0-4f97-bfac-a37f13ac9359 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:22.474844 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4b565c-6f91-4a2d-8d89-4c06b93df7af None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:22.488854 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c8683a-7723-421f-b417-f98a960693c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:22.489496 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c8683a-7723-421f-b417-f98a960693c0 None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:22.489776 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c8683a-7723-421f-b417-f98a960693c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:22.490042 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c8683a-7723-421f-b417-f98a960693c0 None admin] PATH_INFO: `/v3/projects/16ac205ec6f241d69d5e6b8408f09f38/users/dbe4a6e4b6184826b30ac4fc13ad9b1f/roles/5146b19817b54113be3f81111ef9d787` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:22.504678 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c8683a-7723-421f-b417-f98a960693c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=16ac205ec6f241d69d5e6b8408f09f38, user_id=dbe4a6e4b6184826b30ac4fc13ad9b1f, role_id=5146b19817b54113be3f81111ef9d787)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:22.506130 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c8683a-7723-421f-b417-f98a960693c0 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:22.508190 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0c89fe-217d-4739-8943-450fc43a9456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:22.508818 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c89fe-217d-4739-8943-450fc43a9456 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:22.509096 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c89fe-217d-4739-8943-450fc43a9456 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:22.509366 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c89fe-217d-4739-8943-450fc43a9456 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:22.509999 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0c89fe-217d-4739-8943-450fc43a9456 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:22.510297 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0c89fe-217d-4739-8943-450fc43a9456 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:22.512005 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0c89fe-217d-4739-8943-450fc43a9456 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:22.518023 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 197/400] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Oct 27 09:24:22 2022] PUT /identity/v3/projects/16ac205ec6f241d69d5e6b8408f09f38/users/dbe4a6e4b6184826b30ac4fc13ad9b1f/roles/5146b19817b54113be3f81111ef9d787 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:24:22.518797 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c8683a-7723-421f-b417-f98a960693c0 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:22.542859 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 204/401] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:24:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:22.543579 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0c89fe-217d-4739-8943-450fc43a9456 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:22.561247 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:22.647382 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c8683a-7723-421f-b417-f98a960693c0 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:22.659293 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0c89fe-217d-4739-8943-450fc43a9456 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:22.684646 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b59aaf-844c-4715-a8a4-18003f1cba10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:22.685282 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b59aaf-844c-4715-a8a4-18003f1cba10 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:22.685559 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b59aaf-844c-4715-a8a4-18003f1cba10 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:22.685833 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b59aaf-844c-4715-a8a4-18003f1cba10 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:22.693744 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b59aaf-844c-4715-a8a4-18003f1cba10 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:22.695730 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b59aaf-844c-4715-a8a4-18003f1cba10 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:22.700370 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8548cb-f111-42c9-bf24-ee3d5d7e40f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:22.701009 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8548cb-f111-42c9-bf24-ee3d5d7e40f7 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:22.701286 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8548cb-f111-42c9-bf24-ee3d5d7e40f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:22.701557 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8548cb-f111-42c9-bf24-ee3d5d7e40f7 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:22.702284 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8548cb-f111-42c9-bf24-ee3d5d7e40f7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:22.702588 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8548cb-f111-42c9-bf24-ee3d5d7e40f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:22.704020 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8548cb-f111-42c9-bf24-ee3d5d7e40f7 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:22.708799 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-56b59aaf-844c-4715-a8a4-18003f1cba10 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 27 09:24:22.708799 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:22.708799 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:22.708799 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:22.708799 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:22.708799 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:22.708799 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:22.708799 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:22.708799 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:22.708799 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:22.708799 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:22.708799 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:22.708799 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 09:24:22.708799 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 09:24:22.708799 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 09:24:22.708799 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 09:24:22.709851 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:22.709851 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:22.709851 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 09:24:22.709851 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:24:22.709851 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 09:24:22.709851 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:24:22.709851 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:22.709851 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:22.709851 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:22.709851 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:22.709851 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:22.709851 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:22.709851 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:22.709851 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:22.709851 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:22.709851 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:22.709851 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:22.709851 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:22.711406 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:22.711406 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:22.711406 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 09:24:22.711406 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 09:24:22.711406 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 09:24:22.711406 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 09:24:22.711406 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 09:24:22.711406 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 09:24:22.711406 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 27 09:24:22.711406 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:22.711406 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 198/402] 10.222.0.70 () {56 vars in 1102 bytes} [Thu Oct 27 09:24:22 2022] GET /identity/v3/users/neutron => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:22.711957 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b59aaf-844c-4715-a8a4-18003f1cba10 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:22.732554 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:22.752043 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 205/403] 10.222.0.70 () {56 vars in 1137 bytes} [Thu Oct 27 09:24:22 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:22.752043 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8548cb-f111-42c9-bf24-ee3d5d7e40f7 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:22.805121 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:22.816593 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b59aaf-844c-4715-a8a4-18003f1cba10 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:22.851100 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bfff00-5c20-4a54-8c50-204fdfe168ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:22.852232 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bfff00-5c20-4a54-8c50-204fdfe168ef None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:22.852527 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bfff00-5c20-4a54-8c50-204fdfe168ef None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:22.852813 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bfff00-5c20-4a54-8c50-204fdfe168ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:22.857586 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8548cb-f111-42c9-bf24-ee3d5d7e40f7 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:22.859461 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bfff00-5c20-4a54-8c50-204fdfe168ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:22.860943 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bfff00-5c20-4a54-8c50-204fdfe168ef None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:22.872774 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-66bfff00-5c20-4a54-8c50-204fdfe168ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:24:22.872774 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:22.872774 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:22.872774 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:22.872774 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:22.872774 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:22.872774 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:22.872774 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:22.872774 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:22.872774 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:22.872774 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:22.872774 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:22.872774 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 09:24:22.872774 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 09:24:22.872774 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 09:24:22.873655 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 09:24:22.873655 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:22.873655 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:22.873655 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:22.873655 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:22.873655 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:22.873655 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:22.873655 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:22.873655 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:22.873655 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:22.873655 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:22.873655 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:22.873655 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:22.873655 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:22.873655 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:22.873655 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:22.873655 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:22.875180 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 09:24:22.875180 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 09:24:22.875180 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 09:24:22.875180 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 09:24:22.875180 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 09:24:22.875180 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 09:24:22.875180 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:24:22.875180 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:22.875180 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 199/404] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Oct 27 09:24:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:22.875180 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bfff00-5c20-4a54-8c50-204fdfe168ef None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:22.899794 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271b12ab-d8d5-41a5-889f-31419cd0bab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:22.900402 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271b12ab-d8d5-41a5-889f-31419cd0bab7 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:22.900709 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271b12ab-d8d5-41a5-889f-31419cd0bab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:22.900985 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271b12ab-d8d5-41a5-889f-31419cd0bab7 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:22.901633 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271b12ab-d8d5-41a5-889f-31419cd0bab7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:22.903088 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271b12ab-d8d5-41a5-889f-31419cd0bab7 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:22.936755 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 206/405] 10.222.0.70 () {60 vars in 1132 bytes} [Thu Oct 27 09:24:22 2022] POST /identity/v3/users => generated 247 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:24:22.937743 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271b12ab-d8d5-41a5-889f-31419cd0bab7 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:22.998707 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bfff00-5c20-4a54-8c50-204fdfe168ef None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:23.019682 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:23.048866 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311f2fde-c07b-442a-b754-ad92584ece41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:23.049489 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f2fde-c07b-442a-b754-ad92584ece41 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:23.049765 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f2fde-c07b-442a-b754-ad92584ece41 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:23.050035 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f2fde-c07b-442a-b754-ad92584ece41 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:23.051176 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311f2fde-c07b-442a-b754-ad92584ece41 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:23.051479 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311f2fde-c07b-442a-b754-ad92584ece41 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:23.053695 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311f2fde-c07b-442a-b754-ad92584ece41 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:23.080432 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 200/406] 10.222.0.70 () {56 vars in 1139 bytes} [Thu Oct 27 09:24:22 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:23.081329 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311f2fde-c07b-442a-b754-ad92584ece41 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:23.082179 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271b12ab-d8d5-41a5-889f-31419cd0bab7 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:23.095713 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:23.128200 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aec73f-052e-4a56-82d1-349cc2cee636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:23.129185 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aec73f-052e-4a56-82d1-349cc2cee636 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:23.129686 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aec73f-052e-4a56-82d1-349cc2cee636 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:23.130028 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aec73f-052e-4a56-82d1-349cc2cee636 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:23.141468 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aec73f-052e-4a56-82d1-349cc2cee636 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:23.142931 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aec73f-052e-4a56-82d1-349cc2cee636 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:23.159258 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-d0aec73f-052e-4a56-82d1-349cc2cee636 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:23.159258 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:23.159258 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:23.159258 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:23.159258 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:23.159258 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:23.159258 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:23.159258 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:23.159258 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:23.159258 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:23.159258 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:23.159258 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:23.159258 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:24:23.159258 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:24:23.159258 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:24:23.159258 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:24:23.160192 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:23.160192 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:23.160192 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:23.160192 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:23.160192 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:23.160192 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:23.160192 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:23.160192 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:23.160192 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:23.160192 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:23.160192 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:23.160192 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:23.160192 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:23.160192 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:23.160192 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:23.160192 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:23.160192 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:24:23.161077 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:24:23.161077 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:23.161077 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:23.161077 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 207/407] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:24:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:23.161298 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aec73f-052e-4a56-82d1-349cc2cee636 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:23.207234 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311f2fde-c07b-442a-b754-ad92584ece41 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:23.246487 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3172288-1bf3-49a7-b7b5-92ea22797fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:23.247398 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3172288-1bf3-49a7-b7b5-92ea22797fd3 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:23.248964 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3172288-1bf3-49a7-b7b5-92ea22797fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:23.249245 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3172288-1bf3-49a7-b7b5-92ea22797fd3 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:23.250026 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3172288-1bf3-49a7-b7b5-92ea22797fd3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:23.250405 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3172288-1bf3-49a7-b7b5-92ea22797fd3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:23.252493 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3172288-1bf3-49a7-b7b5-92ea22797fd3 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:23.274505 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 201/408] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Oct 27 09:24:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:23.275619 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3172288-1bf3-49a7-b7b5-92ea22797fd3 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:23.291414 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aec73f-052e-4a56-82d1-349cc2cee636 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:23.299156 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:23.336952 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d65bbb-d780-448e-ad82-5db03138c928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:23.337677 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d65bbb-d780-448e-ad82-5db03138c928 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:23.338080 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d65bbb-d780-448e-ad82-5db03138c928 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:23.338350 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d65bbb-d780-448e-ad82-5db03138c928 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:23.352584 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d65bbb-d780-448e-ad82-5db03138c928 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:23.354530 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d65bbb-d780-448e-ad82-5db03138c928 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:23.360777 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:23.373210 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-74d65bbb-d780-448e-ad82-5db03138c928 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:23.373210 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:23.373210 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:23.373210 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:23.373210 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:23.373210 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:23.373210 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:23.373210 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:23.373210 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:23.373210 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:23.373210 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:23.373210 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:23.373210 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:24:23.373210 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:24:23.373210 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:24:23.373210 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:24:23.374264 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:23.374264 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:23.374264 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:23.374264 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:23.374264 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:23.374264 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:23.374264 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:23.374264 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:23.374264 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:23.374264 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:23.374264 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:23.374264 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:23.374264 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:23.374264 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:23.374264 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:23.374264 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:23.374264 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:24:23.375200 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:24:23.375200 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:23.375200 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:23.377527 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 208/409] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:24:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:23.378422 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d65bbb-d780-448e-ad82-5db03138c928 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:23.401786 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3172288-1bf3-49a7-b7b5-92ea22797fd3 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:23.440392 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da255d1-7583-4a8a-90e2-9337b872a711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:23.441214 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da255d1-7583-4a8a-90e2-9337b872a711 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:23.441593 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da255d1-7583-4a8a-90e2-9337b872a711 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:23.441969 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da255d1-7583-4a8a-90e2-9337b872a711 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:23.442742 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da255d1-7583-4a8a-90e2-9337b872a711 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:23.443188 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da255d1-7583-4a8a-90e2-9337b872a711 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:23.444946 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da255d1-7583-4a8a-90e2-9337b872a711 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:23.459050 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 202/410] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:24:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:23.459738 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da255d1-7583-4a8a-90e2-9337b872a711 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:23.500909 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d65bbb-d780-448e-ad82-5db03138c928 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:23.541244 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ded3bf-d4dd-4b95-9ac2-b7ca7b435381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:23.542048 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ded3bf-d4dd-4b95-9ac2-b7ca7b435381 None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:23.542431 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ded3bf-d4dd-4b95-9ac2-b7ca7b435381 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:23.542808 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ded3bf-d4dd-4b95-9ac2-b7ca7b435381 None admin] PATH_INFO: `/v3/projects/16ac205ec6f241d69d5e6b8408f09f38/users/5b8e13ce967449c1b12783183e953579/roles/5146b19817b54113be3f81111ef9d787` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:23.560845 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ded3bf-d4dd-4b95-9ac2-b7ca7b435381 None admin] RBAC: Authorizing `identity:create_grant(project_id=16ac205ec6f241d69d5e6b8408f09f38, user_id=5b8e13ce967449c1b12783183e953579, role_id=5146b19817b54113be3f81111ef9d787)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:23.562535 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ded3bf-d4dd-4b95-9ac2-b7ca7b435381 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:23.570762 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da255d1-7583-4a8a-90e2-9337b872a711 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:23.580558 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 209/411] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Oct 27 09:24:23 2022] PUT /identity/v3/projects/16ac205ec6f241d69d5e6b8408f09f38/users/5b8e13ce967449c1b12783183e953579/roles/5146b19817b54113be3f81111ef9d787 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:24:23.582132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ded3bf-d4dd-4b95-9ac2-b7ca7b435381 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:23.599156 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:23.603082 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:23.613917 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e7b31f-9710-49b2-b2d8-36c4da1a1284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:23.614570 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7b31f-9710-49b2-b2d8-36c4da1a1284 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:23.614851 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7b31f-9710-49b2-b2d8-36c4da1a1284 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:23.615117 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7b31f-9710-49b2-b2d8-36c4da1a1284 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:23.615808 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e7b31f-9710-49b2-b2d8-36c4da1a1284 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:23.616114 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e7b31f-9710-49b2-b2d8-36c4da1a1284 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:23.618094 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e7b31f-9710-49b2-b2d8-36c4da1a1284 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:23.634446 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 203/412] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:24:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:23.641752 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e7b31f-9710-49b2-b2d8-36c4da1a1284 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:23.774664 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e7b31f-9710-49b2-b2d8-36c4da1a1284 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:23.807087 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ded3bf-d4dd-4b95-9ac2-b7ca7b435381 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:23.818518 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9b963a-3df4-47af-b3cd-13edcb066b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:23.819518 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b963a-3df4-47af-b3cd-13edcb066b88 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:23.820629 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b963a-3df4-47af-b3cd-13edcb066b88 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:23.821170 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b963a-3df4-47af-b3cd-13edcb066b88 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:23.830012 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9b963a-3df4-47af-b3cd-13edcb066b88 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:23.832615 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9b963a-3df4-47af-b3cd-13edcb066b88 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:23.848237 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-ae9b963a-3df4-47af-b3cd-13edcb066b88 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:24:23.848237 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:23.848237 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:23.848237 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:23.848237 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:23.848237 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:23.848237 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:23.848237 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:23.848237 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:23.848237 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:23.848237 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:23.848237 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:23.848237 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 09:24:23.848237 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 09:24:23.848237 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 09:24:23.849176 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 09:24:23.849176 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:23.849176 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:23.849176 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:23.849176 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:23.849176 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:23.849176 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:23.849176 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:23.849176 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:23.849176 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:23.849176 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:23.849176 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:23.849176 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:23.849176 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:23.849176 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:23.849176 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:23.849176 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:23.850021 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 09:24:23.850021 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 09:24:23.850021 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 09:24:23.850021 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 09:24:23.850021 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 09:24:23.850021 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 09:24:23.850021 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:24:23.850021 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:23.852684 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 204/413] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Oct 27 09:24:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:23.859331 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9b963a-3df4-47af-b3cd-13edcb066b88 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:23.877661 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733caf91-1d5c-41cb-815f-8b015c77f63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:23.878298 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733caf91-1d5c-41cb-815f-8b015c77f63a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:23.878575 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733caf91-1d5c-41cb-815f-8b015c77f63a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:23.878843 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733caf91-1d5c-41cb-815f-8b015c77f63a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:23.886821 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733caf91-1d5c-41cb-815f-8b015c77f63a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:23.888268 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733caf91-1d5c-41cb-815f-8b015c77f63a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:23.902373 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-733caf91-1d5c-41cb-815f-8b015c77f63a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:24:23.902373 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:23.902373 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:23.902373 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:23.902373 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:23.902373 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:23.902373 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:23.902373 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:23.902373 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:23.902373 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:23.902373 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:23.902373 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:23.902373 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 09:24:23.902373 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 09:24:23.902373 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 09:24:23.903283 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 09:24:23.903283 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:23.903283 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:23.903283 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:23.903283 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:23.903283 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:23.903283 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:23.903283 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:23.903283 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:23.903283 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:23.903283 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:23.903283 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:23.903283 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:23.903283 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:23.903283 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:23.903283 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:23.903283 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:23.904158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 09:24:23.904158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 09:24:23.904158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 09:24:23.904158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 09:24:23.904158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 09:24:23.904158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 09:24:23.904158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:24:23.904158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:23.904158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 210/414] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Oct 27 09:24:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:23.909483 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733caf91-1d5c-41cb-815f-8b015c77f63a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:23.928935 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:23.993079 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9b963a-3df4-47af-b3cd-13edcb066b88 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:24.009020 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:24.030880 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3c4a35-e0d1-4fa8-b0a2-15ad462806af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:24.031722 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c4a35-e0d1-4fa8-b0a2-15ad462806af None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:24.031832 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c4a35-e0d1-4fa8-b0a2-15ad462806af None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:24.032017 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c4a35-e0d1-4fa8-b0a2-15ad462806af None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:24.032731 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3c4a35-e0d1-4fa8-b0a2-15ad462806af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:24.033034 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3c4a35-e0d1-4fa8-b0a2-15ad462806af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:24.034469 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3c4a35-e0d1-4fa8-b0a2-15ad462806af None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:24.047247 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 205/415] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Oct 27 09:24:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:24.052854 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3c4a35-e0d1-4fa8-b0a2-15ad462806af None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:24.053815 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733caf91-1d5c-41cb-815f-8b015c77f63a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:24.091221 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7067a70-0326-4f81-b289-df4375e4af00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:24.091859 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7067a70-0326-4f81-b289-df4375e4af00 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:24.092125 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7067a70-0326-4f81-b289-df4375e4af00 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:24.092312 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7067a70-0326-4f81-b289-df4375e4af00 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:24.093111 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7067a70-0326-4f81-b289-df4375e4af00 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:24.093416 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7067a70-0326-4f81-b289-df4375e4af00 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:24.095202 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7067a70-0326-4f81-b289-df4375e4af00 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:24.113292 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 211/416] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Oct 27 09:24:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:24:24.121877 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7067a70-0326-4f81-b289-df4375e4af00 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:24.209208 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3c4a35-e0d1-4fa8-b0a2-15ad462806af None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:24.219080 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:24.247761 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae1c4c-73e5-461f-884f-e3df21063bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:24.248480 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae1c4c-73e5-461f-884f-e3df21063bed None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:24.248783 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae1c4c-73e5-461f-884f-e3df21063bed None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:24.249056 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae1c4c-73e5-461f-884f-e3df21063bed None admin] PATH_INFO: `/v3/projects/16ac205ec6f241d69d5e6b8408f09f38/users/45d8920d9a7940abb112dbbb3d7f6e65/roles/5146b19817b54113be3f81111ef9d787` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:24.265206 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ae1c4c-73e5-461f-884f-e3df21063bed None admin] RBAC: Authorizing `identity:create_grant(project_id=16ac205ec6f241d69d5e6b8408f09f38, user_id=45d8920d9a7940abb112dbbb3d7f6e65, role_id=5146b19817b54113be3f81111ef9d787)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:24.266678 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ae1c4c-73e5-461f-884f-e3df21063bed None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:24.301378 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7067a70-0326-4f81-b289-df4375e4af00 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:24.306937 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 206/417] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Oct 27 09:24:24 2022] PUT /identity/v3/projects/16ac205ec6f241d69d5e6b8408f09f38/users/45d8920d9a7940abb112dbbb3d7f6e65/roles/5146b19817b54113be3f81111ef9d787 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:24:24.340053 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51dea1-193d-4ee8-a47a-295391761f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:24.340761 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51dea1-193d-4ee8-a47a-295391761f43 None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:24.341089 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51dea1-193d-4ee8-a47a-295391761f43 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:24.341369 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51dea1-193d-4ee8-a47a-295391761f43 None admin] PATH_INFO: `/v3/projects/16ac205ec6f241d69d5e6b8408f09f38/users/72b0538f0e42446887b1d95b2fb41fb2/roles/5146b19817b54113be3f81111ef9d787` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:24.357458 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:24.364945 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c51dea1-193d-4ee8-a47a-295391761f43 None admin] RBAC: Authorizing `identity:create_grant(project_id=16ac205ec6f241d69d5e6b8408f09f38, user_id=72b0538f0e42446887b1d95b2fb41fb2, role_id=5146b19817b54113be3f81111ef9d787)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:24.366539 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c51dea1-193d-4ee8-a47a-295391761f43 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:24.406137 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 212/418] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Oct 27 09:24:24 2022] PUT /identity/v3/projects/16ac205ec6f241d69d5e6b8408f09f38/users/72b0538f0e42446887b1d95b2fb41fb2/roles/5146b19817b54113be3f81111ef9d787 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:24:24.970880 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19cbeb7-08ba-4241-868a-f3bef1b394bc None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:24.971273 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19cbeb7-08ba-4241-868a-f3bef1b394bc None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:24.971474 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19cbeb7-08ba-4241-868a-f3bef1b394bc None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:24.973165 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 207/419] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:24.978352 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d7bc7-2e90-42fd-8ee8-389af1a27127 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:24.978634 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d7bc7-2e90-42fd-8ee8-389af1a27127 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:24.978899 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d7bc7-2e90-42fd-8ee8-389af1a27127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:25.001562 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-c05d7bc7-2e90-42fd-8ee8-389af1a27127 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:25.002777 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c05d7bc7-2e90-42fd-8ee8-389af1a27127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:25.089706 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 213/420] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:24:25.467943 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19cbeb7-08ba-4241-868a-f3bef1b394bc None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:25.569340 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c8f318-6c45-456f-9d85-e508cea2652f None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:25.569879 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c8f318-6c45-456f-9d85-e508cea2652f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:25.569879 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c8f318-6c45-456f-9d85-e508cea2652f None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:25.571516 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 214/421] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:25.576298 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e84ab70-475c-4ace-9b30-ed580590721c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:25.576611 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e84ab70-475c-4ace-9b30-ed580590721c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:25.576887 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e84ab70-475c-4ace-9b30-ed580590721c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:25.587387 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19cbeb7-08ba-4241-868a-f3bef1b394bc None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:25.609188 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-4e84ab70-475c-4ace-9b30-ed580590721c None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:25.610386 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4e84ab70-475c-4ace-9b30-ed580590721c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:25.654009 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da44c0bb-fcf3-4110-8395-a8c6ed2edc79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:25.654636 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44c0bb-fcf3-4110-8395-a8c6ed2edc79 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:25.654914 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44c0bb-fcf3-4110-8395-a8c6ed2edc79 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:25.655182 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44c0bb-fcf3-4110-8395-a8c6ed2edc79 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:25.675572 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da44c0bb-fcf3-4110-8395-a8c6ed2edc79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:24:25.687923 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-da44c0bb-fcf3-4110-8395-a8c6ed2edc79 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 27 09:24:25.687923 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:25.687923 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:25.687923 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:25.687923 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:25.687923 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:25.687923 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:25.687923 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:25.687923 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:25.687923 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:25.687923 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:25.687923 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:25.687923 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:24:25.687923 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:24:25.687923 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:24:25.687923 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:24:25.688852 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:24:25.688852 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:24:25.688852 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:24:25.688852 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:24:25.688852 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:25.688852 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:25.688852 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:25.688852 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:25.688852 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:25.688852 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:25.688852 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:25.688852 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:25.688852 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:25.688852 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:25.688852 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:25.688852 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:25.690299 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:25.690299 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:25.690299 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:25.690299 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:25.690299 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:24:25.690299 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:24:25.690299 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:24:25.690299 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:24:25.690299 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:24:25.690299 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:24:25.690299 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 27 09:24:25.690299 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:25.690299 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 208/422] 10.222.0.70 () {56 vars in 1102 bytes} [Thu Oct 27 09:24:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:25.695139 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da44c0bb-fcf3-4110-8395-a8c6ed2edc79 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:25.735659 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:25.753850 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 215/423] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:24:25.823322 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da44c0bb-fcf3-4110-8395-a8c6ed2edc79 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:25.874597 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5896b011-2a6c-4b62-91c6-8c635782193c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:25.875274 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896b011-2a6c-4b62-91c6-8c635782193c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:25.875555 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896b011-2a6c-4b62-91c6-8c635782193c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:25.875827 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896b011-2a6c-4b62-91c6-8c635782193c None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:25.876560 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5896b011-2a6c-4b62-91c6-8c635782193c None admin] RBAC: Adding query filter params (name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:25.876871 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5896b011-2a6c-4b62-91c6-8c635782193c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:25.878250 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5896b011-2a6c-4b62-91c6-8c635782193c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:25.891223 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 209/424] 10.222.0.70 () {56 vars in 1103 bytes} [Thu Oct 27 09:24:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:25.897054 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e84ab70-475c-4ace-9b30-ed580590721c None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:25.964444 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:26.048491 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e84ab70-475c-4ace-9b30-ed580590721c None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:26.092656 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0827ecff-eace-4fbc-8f78-3119d178897f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:26.093628 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0827ecff-eace-4fbc-8f78-3119d178897f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:26.094102 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0827ecff-eace-4fbc-8f78-3119d178897f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:26.094827 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0827ecff-eace-4fbc-8f78-3119d178897f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:26.109096 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0827ecff-eace-4fbc-8f78-3119d178897f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:26.110766 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0827ecff-eace-4fbc-8f78-3119d178897f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:26.119059 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5896b011-2a6c-4b62-91c6-8c635782193c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:26.137368 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-0827ecff-eace-4fbc-8f78-3119d178897f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:26.137368 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:26.137368 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:26.137368 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:26.137368 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:26.137368 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:26.137368 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:26.137368 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:26.137368 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:26.137368 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:26.137368 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:26.137368 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:26.137368 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:24:26.137368 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:24:26.137368 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:24:26.137368 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:24:26.138346 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:26.138346 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:26.138346 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:26.138346 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:26.138346 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:26.138346 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:26.138346 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:26.138346 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:26.138346 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:26.138346 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:26.138346 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:26.138346 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:26.138346 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:26.138346 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:26.138346 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:26.138346 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:26.138346 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:24:26.139193 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:24:26.139193 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:26.139193 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:26.141524 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 216/425] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:24:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:26.146805 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0827ecff-eace-4fbc-8f78-3119d178897f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:26.217834 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:26.241280 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5896b011-2a6c-4b62-91c6-8c635782193c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:26.286915 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:26.297882 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0827ecff-eace-4fbc-8f78-3119d178897f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:26.300357 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d500423f-d744-40cc-8fa7-1139cac6d03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:26.301031 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d500423f-d744-40cc-8fa7-1139cac6d03d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:26.301305 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d500423f-d744-40cc-8fa7-1139cac6d03d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:26.301572 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d500423f-d744-40cc-8fa7-1139cac6d03d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:26.312884 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d500423f-d744-40cc-8fa7-1139cac6d03d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:24:26.318538 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-d500423f-d744-40cc-8fa7-1139cac6d03d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 27 09:24:26.318538 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:26.318538 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:26.318538 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:26.318538 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:26.318538 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:26.318538 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:26.318538 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:26.318538 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:26.318538 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:26.318538 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:26.318538 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:26.318538 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:24:26.318538 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:24:26.318538 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:24:26.318538 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:24:26.319422 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:24:26.319422 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:24:26.319422 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:24:26.319422 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:24:26.319422 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:26.319422 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:26.319422 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:26.319422 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:26.319422 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:26.319422 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:26.319422 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:26.319422 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:26.319422 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:26.319422 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:26.319422 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:26.319422 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:26.320783 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:26.320783 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:26.320783 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:26.320783 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:26.320783 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:24:26.320783 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:24:26.320783 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:24:26.320783 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:24:26.320783 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:24:26.320783 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:24:26.320783 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 27 09:24:26.320783 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:26.325376 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 210/426] 10.222.0.70 () {56 vars in 1102 bytes} [Thu Oct 27 09:24:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:26.326729 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7911d9b-e000-490e-9091-ca0e92cad201 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:26.327014 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7911d9b-e000-490e-9091-ca0e92cad201 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:26.327289 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7911d9b-e000-490e-9091-ca0e92cad201 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:26.328943 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 211/427] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:26.329760 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7911d9b-e000-490e-9091-ca0e92cad201 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:26.334461 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72c8b93-9859-4602-82a9-03d6706fa0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:26.335062 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c8b93-9859-4602-82a9-03d6706fa0b2 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:26.335400 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c8b93-9859-4602-82a9-03d6706fa0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:26.336349 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c8b93-9859-4602-82a9-03d6706fa0b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:26.337027 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72c8b93-9859-4602-82a9-03d6706fa0b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:26.337573 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72c8b93-9859-4602-82a9-03d6706fa0b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:26.339858 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72c8b93-9859-4602-82a9-03d6706fa0b2 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:26.354987 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 217/428] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:24:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:24:26.356611 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ff67a-6b79-401f-a101-c0d0af6e252a None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:26.356892 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ff67a-6b79-401f-a101-c0d0af6e252a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:26.357160 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ff67a-6b79-401f-a101-c0d0af6e252a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:26.378115 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-8e0ff67a-6b79-401f-a101-c0d0af6e252a None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:26.379704 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8e0ff67a-6b79-401f-a101-c0d0af6e252a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:26.447429 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7911d9b-e000-490e-9091-ca0e92cad201 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:26.455704 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 218/429] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:24:26.456647 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0ff67a-6b79-401f-a101-c0d0af6e252a None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:26.517978 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275d2ee6-361b-4cf0-a425-864acf75e7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:26.518682 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d2ee6-361b-4cf0-a425-864acf75e7fa None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:26.518967 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d2ee6-361b-4cf0-a425-864acf75e7fa None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:26.519237 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d2ee6-361b-4cf0-a425-864acf75e7fa None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:26.519901 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275d2ee6-361b-4cf0-a425-864acf75e7fa None admin] RBAC: Adding query filter params (name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:26.520306 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275d2ee6-361b-4cf0-a425-864acf75e7fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:26.521813 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275d2ee6-361b-4cf0-a425-864acf75e7fa None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:26.530859 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 212/430] 10.222.0.70 () {56 vars in 1103 bytes} [Thu Oct 27 09:24:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:26.536353 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275d2ee6-361b-4cf0-a425-864acf75e7fa None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:26.539240 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:26.561642 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:26.603601 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0ff67a-6b79-401f-a101-c0d0af6e252a None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:26.674101 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649440ce-6d17-4eda-bb9d-660710c92cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:26.674658 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649440ce-6d17-4eda-bb9d-660710c92cbd None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:26.674935 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649440ce-6d17-4eda-bb9d-660710c92cbd None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:26.675202 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649440ce-6d17-4eda-bb9d-660710c92cbd None admin] PATH_INFO: `/v3/users/nova` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:26.697070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649440ce-6d17-4eda-bb9d-660710c92cbd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:26.698541 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649440ce-6d17-4eda-bb9d-660710c92cbd None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:26.718321 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-649440ce-6d17-4eda-bb9d-660710c92cbd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 27 09:24:26.718321 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:26.718321 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:26.718321 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:26.718321 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:26.718321 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:26.718321 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:26.718321 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:26.718321 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:26.718321 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:26.718321 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:26.718321 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:26.718321 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 09:24:26.718321 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 09:24:26.718321 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 09:24:26.718321 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 09:24:26.720813 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:26.720813 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:26.720813 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 09:24:26.720813 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:24:26.720813 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 09:24:26.720813 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:24:26.720813 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:26.720813 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:26.720813 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:26.720813 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:26.720813 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:26.720813 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:26.720813 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:26.720813 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:26.720813 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:26.720813 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:26.720813 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:26.720813 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:26.723039 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:26.723039 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:26.723039 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 09:24:26.723039 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 09:24:26.723039 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 09:24:26.723039 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 09:24:26.723039 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 09:24:26.723039 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 09:24:26.723039 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 27 09:24:26.723039 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:26.723039 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 219/431] 10.222.0.70 () {56 vars in 1093 bytes} [Thu Oct 27 09:24:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:26.729654 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649440ce-6d17-4eda-bb9d-660710c92cbd None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:26.766722 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275d2ee6-361b-4cf0-a425-864acf75e7fa None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:26.817366 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d261642b-1a6c-41af-83d1-33e141d2f765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:26.818003 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d261642b-1a6c-41af-83d1-33e141d2f765 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:26.818326 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d261642b-1a6c-41af-83d1-33e141d2f765 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:26.818695 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d261642b-1a6c-41af-83d1-33e141d2f765 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:26.830384 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d261642b-1a6c-41af-83d1-33e141d2f765 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:26.831876 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d261642b-1a6c-41af-83d1-33e141d2f765 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:26.851614 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-d261642b-1a6c-41af-83d1-33e141d2f765 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:26.851614 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:26.851614 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:26.851614 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:26.851614 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:26.851614 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:26.851614 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:26.851614 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:26.851614 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:26.851614 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:26.851614 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:26.851614 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:26.851614 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:24:26.851614 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:24:26.851614 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:24:26.851614 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:24:26.853895 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:26.853895 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:26.853895 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:26.853895 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:26.853895 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:26.853895 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:26.853895 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:26.853895 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:26.853895 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:26.853895 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:26.853895 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:26.853895 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:26.853895 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:26.853895 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:26.853895 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:26.853895 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:26.853895 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:24:26.856089 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:24:26.856089 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:26.856089 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:26.856089 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 213/432] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:24:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 09:24:26.856089 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d261642b-1a6c-41af-83d1-33e141d2f765 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:26.878547 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649440ce-6d17-4eda-bb9d-660710c92cbd None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:26.917299 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:26.919309 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef9d65f-ed02-48f0-9176-b8799def6618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:26.920013 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9d65f-ed02-48f0-9176-b8799def6618 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:26.920350 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9d65f-ed02-48f0-9176-b8799def6618 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:26.920701 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9d65f-ed02-48f0-9176-b8799def6618 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:26.921419 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef9d65f-ed02-48f0-9176-b8799def6618 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:26.921910 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef9d65f-ed02-48f0-9176-b8799def6618 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:26.923125 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef9d65f-ed02-48f0-9176-b8799def6618 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:26.934180 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:26.943236 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 220/433] 10.222.0.70 () {56 vars in 1133 bytes} [Thu Oct 27 09:24:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:26.944438 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef9d65f-ed02-48f0-9176-b8799def6618 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:26.968966 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d261642b-1a6c-41af-83d1-33e141d2f765 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:27.005195 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06ec58a-e3ca-43ba-8d74-33507e887f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:27.006060 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ec58a-e3ca-43ba-8d74-33507e887f90 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:27.006508 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ec58a-e3ca-43ba-8d74-33507e887f90 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:27.006776 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ec58a-e3ca-43ba-8d74-33507e887f90 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:27.016991 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06ec58a-e3ca-43ba-8d74-33507e887f90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:24:27.022219 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-d06ec58a-e3ca-43ba-8d74-33507e887f90 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 27 09:24:27.022219 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:27.022219 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:27.022219 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:27.022219 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:27.022219 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:27.022219 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:27.022219 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:27.022219 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:27.022219 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:27.022219 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:27.022219 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:27.022219 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:24:27.022219 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:24:27.022219 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:24:27.022219 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:24:27.023268 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:24:27.023268 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:24:27.023268 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:24:27.023268 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:24:27.023268 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:27.023268 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:27.023268 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:27.023268 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:27.023268 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:27.023268 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:27.023268 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:27.023268 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:27.023268 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:27.023268 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:27.023268 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:27.023268 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:27.024065 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:27.024065 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:27.024065 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:27.024065 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:27.024065 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:24:27.024065 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:24:27.024065 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:24:27.024065 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:24:27.024065 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:24:27.024065 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:24:27.024065 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 27 09:24:27.024065 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:27.028084 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 214/434] 10.222.0.70 () {56 vars in 1102 bytes} [Thu Oct 27 09:24:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:27.029408 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b97210-65d8-44b6-9d27-32e1ef8580a5 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:27.029685 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b97210-65d8-44b6-9d27-32e1ef8580a5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:27.029954 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b97210-65d8-44b6-9d27-32e1ef8580a5 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:27.031443 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 215/435] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:27.031933 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b97210-65d8-44b6-9d27-32e1ef8580a5 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:27.055166 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef9d65f-ed02-48f0-9176-b8799def6618 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:27.093160 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ea80d9-e944-4077-9707-2ce80b2f13c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:27.093881 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea80d9-e944-4077-9707-2ce80b2f13c8 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:27.094224 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea80d9-e944-4077-9707-2ce80b2f13c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:27.094818 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea80d9-e944-4077-9707-2ce80b2f13c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:27.095519 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ea80d9-e944-4077-9707-2ce80b2f13c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:27.095891 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ea80d9-e944-4077-9707-2ce80b2f13c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:27.097350 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ea80d9-e944-4077-9707-2ce80b2f13c8 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:27.109512 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 221/436] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:24:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:27.114688 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ea80d9-e944-4077-9707-2ce80b2f13c8 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:27.139857 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b97210-65d8-44b6-9d27-32e1ef8580a5 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:27.156183 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:27.178611 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:27.181198 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37928dca-432f-4b96-9107-762536fff9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:27.181789 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37928dca-432f-4b96-9107-762536fff9bf None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:27.182078 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37928dca-432f-4b96-9107-762536fff9bf None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:27.182333 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37928dca-432f-4b96-9107-762536fff9bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:27.225506 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ea80d9-e944-4077-9707-2ce80b2f13c8 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:27.254358 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37928dca-432f-4b96-9107-762536fff9bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:27.255826 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37928dca-432f-4b96-9107-762536fff9bf None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:27.260413 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe8f887-d8a2-4be6-a26b-eaf8b486033e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:27.261051 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8f887-d8a2-4be6-a26b-eaf8b486033e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:27.261331 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8f887-d8a2-4be6-a26b-eaf8b486033e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:27.261603 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8f887-d8a2-4be6-a26b-eaf8b486033e None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:27.262254 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe8f887-d8a2-4be6-a26b-eaf8b486033e None admin] RBAC: Adding query filter params (name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:27.262557 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe8f887-d8a2-4be6-a26b-eaf8b486033e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:27.263908 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe8f887-d8a2-4be6-a26b-eaf8b486033e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:27.274788 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 222/437] 10.222.0.70 () {56 vars in 1103 bytes} [Thu Oct 27 09:24:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:27.276330 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2d32b-1426-4b7a-83b7-eb4cddc43af7 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:27.276849 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2d32b-1426-4b7a-83b7-eb4cddc43af7 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:27.277324 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2d32b-1426-4b7a-83b7-eb4cddc43af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:27.297459 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-83a2d32b-1426-4b7a-83b7-eb4cddc43af7 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:27.298637 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-83a2d32b-1426-4b7a-83b7-eb4cddc43af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:27.363020 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 216/438] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Oct 27 09:24:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 27 09:24:27.363852 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37928dca-432f-4b96-9107-762536fff9bf None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:27.368384 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 223/439] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:24:27.369248 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a2d32b-1426-4b7a-83b7-eb4cddc43af7 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:27.394069 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:27.421179 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:27.477861 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a2d32b-1426-4b7a-83b7-eb4cddc43af7 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:27.482404 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37928dca-432f-4b96-9107-762536fff9bf None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:27.513755 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1cae67-9c66-473b-aa9e-fb43cbf44751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:27.514377 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1cae67-9c66-473b-aa9e-fb43cbf44751 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:27.514657 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1cae67-9c66-473b-aa9e-fb43cbf44751 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:27.514923 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1cae67-9c66-473b-aa9e-fb43cbf44751 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:27.516946 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e3489c-051e-4d27-b0fe-a90981e3ccf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:27.517568 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3489c-051e-4d27-b0fe-a90981e3ccf8 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:27.517841 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3489c-051e-4d27-b0fe-a90981e3ccf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:27.518111 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3489c-051e-4d27-b0fe-a90981e3ccf8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:27.525691 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1cae67-9c66-473b-aa9e-fb43cbf44751 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:27.525890 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e3489c-051e-4d27-b0fe-a90981e3ccf8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:27.527135 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1cae67-9c66-473b-aa9e-fb43cbf44751 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:27.527309 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e3489c-051e-4d27-b0fe-a90981e3ccf8 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:27.539396 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-b6e3489c-051e-4d27-b0fe-a90981e3ccf8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 27 09:24:27.539396 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:27.539396 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:27.539396 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:27.539396 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:27.539396 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:27.539396 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:27.539396 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:27.539396 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:27.539396 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:27.539396 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:27.539396 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:27.539396 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 09:24:27.539396 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 09:24:27.539396 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 09:24:27.539396 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 09:24:27.540290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:27.540290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:27.540290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 09:24:27.540290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:24:27.540290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 09:24:27.540290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:24:27.540290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:27.540290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:27.540290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:27.540290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:27.540290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:27.540290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:27.540290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:27.540290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:27.540290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:27.540290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:27.540290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:27.540290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:27.541551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:27.541551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:27.541551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 09:24:27.541551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 09:24:27.541551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 09:24:27.541551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 09:24:27.541551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 09:24:27.541551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 09:24:27.541551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 27 09:24:27.541551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:27.541551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 217/440] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Oct 27 09:24:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:27.542125 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a6e49-8f4a-405c-9593-3d0df94520e6 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:27.542125 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a6e49-8f4a-405c-9593-3d0df94520e6 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:27.542125 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a6e49-8f4a-405c-9593-3d0df94520e6 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:27.547344 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 218/441] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:27.547344 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a6e49-8f4a-405c-9593-3d0df94520e6 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:27.547543 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-ba1cae67-9c66-473b-aa9e-fb43cbf44751 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:27.547543 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:27.547543 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:27.547543 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:27.547543 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:27.547543 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:27.547543 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:27.547543 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:27.547543 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:27.547543 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:27.547543 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:27.547543 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:27.547543 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:24:27.547543 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:24:27.547543 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:24:27.547543 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:24:27.548365 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:27.548365 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:27.548365 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:27.548365 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:27.548365 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:27.548365 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:27.548365 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:27.548365 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:27.548365 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:27.548365 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:27.548365 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:27.548365 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:27.548365 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:27.548365 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:27.548365 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:27.548365 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:27.548365 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:24:27.549275 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:24:27.549275 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:27.549275 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:27.552671 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 224/442] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:24:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:27.553505 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1cae67-9c66-473b-aa9e-fb43cbf44751 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:27.625152 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:27.652943 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a6e49-8f4a-405c-9593-3d0df94520e6 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:27.661786 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1cae67-9c66-473b-aa9e-fb43cbf44751 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:27.663308 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:27.687002 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff264a3-a985-4c9d-8ed7-7139b4d1b6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:27.687621 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff264a3-a985-4c9d-8ed7-7139b4d1b6a5 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:27.687899 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff264a3-a985-4c9d-8ed7-7139b4d1b6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:27.688166 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff264a3-a985-4c9d-8ed7-7139b4d1b6a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:27.695971 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcfc3d1-2b3c-40e0-be05-01408a64b9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:27.696618 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcfc3d1-2b3c-40e0-be05-01408a64b9e6 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:27.696867 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcfc3d1-2b3c-40e0-be05-01408a64b9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:27.697132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcfc3d1-2b3c-40e0-be05-01408a64b9e6 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:27.697836 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcfc3d1-2b3c-40e0-be05-01408a64b9e6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:27.698140 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcfc3d1-2b3c-40e0-be05-01408a64b9e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:27.698562 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff264a3-a985-4c9d-8ed7-7139b4d1b6a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:27.699573 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcfc3d1-2b3c-40e0-be05-01408a64b9e6 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:27.699992 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff264a3-a985-4c9d-8ed7-7139b4d1b6a5 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:27.714965 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-7ff264a3-a985-4c9d-8ed7-7139b4d1b6a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:27.714965 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:27.714965 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:27.714965 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:27.714965 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:27.714965 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:27.714965 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:27.714965 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:27.714965 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:27.714965 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:27.714965 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:27.714965 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:27.714965 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:24:27.714965 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:24:27.714965 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:24:27.714965 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:24:27.716004 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:27.716004 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:27.716004 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:27.716004 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:27.716004 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:27.716004 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:27.716004 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:27.716004 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:27.716004 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:27.716004 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:27.716004 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:27.716004 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:27.716004 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:27.716004 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:27.716004 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:27.716004 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:27.716004 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:24:27.717032 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:24:27.717032 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:27.717032 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:27.717032 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 219/443] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:24:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:27.717291 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d016852b-3f25-4b37-bb30-17bebaa396b6 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:27.717507 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d016852b-3f25-4b37-bb30-17bebaa396b6 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:27.717776 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d016852b-3f25-4b37-bb30-17bebaa396b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:27.722403 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 225/444] 10.222.0.70 () {56 vars in 1135 bytes} [Thu Oct 27 09:24:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:27.723237 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcfc3d1-2b3c-40e0-be05-01408a64b9e6 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:27.735529 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-d016852b-3f25-4b37-bb30-17bebaa396b6 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:27.736675 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d016852b-3f25-4b37-bb30-17bebaa396b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:27.801368 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 220/445] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:24:27.802199 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d016852b-3f25-4b37-bb30-17bebaa396b6 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:27.825596 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcfc3d1-2b3c-40e0-be05-01408a64b9e6 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:27.859890 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:27.867065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abae949a-89f9-481f-8427-b8a20342c0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:27.867668 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae949a-89f9-481f-8427-b8a20342c0b5 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:27.867943 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae949a-89f9-481f-8427-b8a20342c0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:27.868207 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae949a-89f9-481f-8427-b8a20342c0b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:27.868870 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abae949a-89f9-481f-8427-b8a20342c0b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:27.869167 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abae949a-89f9-481f-8427-b8a20342c0b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:27.870636 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abae949a-89f9-481f-8427-b8a20342c0b5 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:27.887367 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 226/446] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:24:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:27.887367 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abae949a-89f9-481f-8427-b8a20342c0b5 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:27.909930 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:27.912351 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d016852b-3f25-4b37-bb30-17bebaa396b6 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:27.953769 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc0806-30c5-4a50-ab14-f3cc01a3297c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:27.954609 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc0806-30c5-4a50-ab14-f3cc01a3297c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:27.954889 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc0806-30c5-4a50-ab14-f3cc01a3297c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:27.955159 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc0806-30c5-4a50-ab14-f3cc01a3297c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:27.966792 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fc0806-30c5-4a50-ab14-f3cc01a3297c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:24:27.972959 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-d9fc0806-30c5-4a50-ab14-f3cc01a3297c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 27 09:24:27.972959 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:27.972959 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:27.972959 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:27.972959 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:27.972959 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:27.972959 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:27.972959 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:27.972959 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:27.972959 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:27.972959 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:27.972959 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:27.972959 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:24:27.972959 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:24:27.972959 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:24:27.972959 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:24:27.974049 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:24:27.974049 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:24:27.974049 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:24:27.974049 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:24:27.974049 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:27.974049 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:27.974049 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:27.974049 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:27.974049 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:27.974049 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:27.974049 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:27.974049 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:27.974049 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:27.974049 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:27.974049 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:27.974049 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:27.975043 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:27.975043 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:27.975043 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:27.975043 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:27.975043 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:24:27.975043 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:24:27.975043 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:24:27.975043 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:24:27.975043 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:24:27.975043 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:24:27.975043 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 27 09:24:27.975043 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:27.975043 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 221/447] 10.222.0.70 () {56 vars in 1102 bytes} [Thu Oct 27 09:24:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:27.975868 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc0806-30c5-4a50-ab14-f3cc01a3297c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:27.996140 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abae949a-89f9-481f-8427-b8a20342c0b5 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:28.032054 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b07fe3-0d30-4ab1-a888-835e14a1baa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:28.032776 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b07fe3-0d30-4ab1-a888-835e14a1baa2 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:28.033267 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b07fe3-0d30-4ab1-a888-835e14a1baa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:28.033609 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b07fe3-0d30-4ab1-a888-835e14a1baa2 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:28.034312 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b07fe3-0d30-4ab1-a888-835e14a1baa2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:28.034615 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b07fe3-0d30-4ab1-a888-835e14a1baa2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:28.035965 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b07fe3-0d30-4ab1-a888-835e14a1baa2 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:28.048291 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 227/448] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:24:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:28.048960 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b07fe3-0d30-4ab1-a888-835e14a1baa2 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:28.085850 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc0806-30c5-4a50-ab14-f3cc01a3297c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:28.102041 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:28.121031 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c81d50-639d-437f-9457-122ee74c2c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:28.121638 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c81d50-639d-437f-9457-122ee74c2c59 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:28.121908 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c81d50-639d-437f-9457-122ee74c2c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:28.122170 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c81d50-639d-437f-9457-122ee74c2c59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:28.130168 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:28.132821 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c81d50-639d-437f-9457-122ee74c2c59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:28.134237 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c81d50-639d-437f-9457-122ee74c2c59 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:28.148338 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-f4c81d50-639d-437f-9457-122ee74c2c59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:28.148338 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:28.148338 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:28.148338 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:28.148338 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:28.148338 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:28.148338 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:28.148338 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:28.148338 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:28.148338 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:28.148338 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:28.148338 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:28.148338 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:24:28.148338 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:24:28.148338 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:24:28.148338 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:24:28.149366 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:28.149366 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:28.149366 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:28.149366 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:28.149366 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:28.149366 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:28.149366 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:28.149366 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:28.149366 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:28.149366 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:28.149366 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:28.149366 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:28.149366 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:28.149366 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:28.149366 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:28.149366 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:28.149366 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:24:28.150211 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:24:28.150211 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:28.150211 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:28.150211 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 222/449] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:24:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:28.150436 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b07fe3-0d30-4ab1-a888-835e14a1baa2 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:28.150506 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c81d50-639d-437f-9457-122ee74c2c59 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:28.183687 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261622c0-7724-49b9-be6e-afa74b04bfd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:28.184304 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261622c0-7724-49b9-be6e-afa74b04bfd2 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:28.184612 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261622c0-7724-49b9-be6e-afa74b04bfd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:28.184910 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261622c0-7724-49b9-be6e-afa74b04bfd2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:28.192801 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261622c0-7724-49b9-be6e-afa74b04bfd2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:28.194223 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261622c0-7724-49b9-be6e-afa74b04bfd2 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:28.206188 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-261622c0-7724-49b9-be6e-afa74b04bfd2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 27 09:24:28.206188 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:28.206188 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:28.206188 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:28.206188 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:28.206188 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:28.206188 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:28.206188 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:28.206188 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:28.206188 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:28.206188 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:28.206188 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:28.206188 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 09:24:28.206188 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 09:24:28.206188 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 09:24:28.206188 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 09:24:28.207069 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:28.207069 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:28.207069 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 09:24:28.207069 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:24:28.207069 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 09:24:28.207069 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:24:28.207069 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:28.207069 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:28.207069 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:28.207069 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:28.207069 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:28.207069 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:28.207069 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:28.207069 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:28.207069 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:28.207069 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:28.207069 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:28.207069 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:28.208010 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:28.208010 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:28.208010 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 09:24:28.208010 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 09:24:28.208010 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 09:24:28.208010 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 09:24:28.208010 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 09:24:28.208010 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 09:24:28.208010 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 27 09:24:28.208010 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:28.208010 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 228/450] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Oct 27 09:24:28 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:28.208010 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261622c0-7724-49b9-be6e-afa74b04bfd2 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:28.253860 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c81d50-639d-437f-9457-122ee74c2c59 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:28.289486 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb853a-2a8c-4bba-8631-2076546a588a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:28.290082 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb853a-2a8c-4bba-8631-2076546a588a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:28.290355 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb853a-2a8c-4bba-8631-2076546a588a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:28.290620 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb853a-2a8c-4bba-8631-2076546a588a None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:28.291262 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceb853a-2a8c-4bba-8631-2076546a588a None admin] RBAC: Adding query filter params (name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:28.291558 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceb853a-2a8c-4bba-8631-2076546a588a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:28.292963 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceb853a-2a8c-4bba-8631-2076546a588a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:28.300848 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 223/451] 10.222.0.70 () {56 vars in 1103 bytes} [Thu Oct 27 09:24:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:28.301527 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb853a-2a8c-4bba-8631-2076546a588a None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:28.311526 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261622c0-7724-49b9-be6e-afa74b04bfd2 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:28.330952 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:28.344909 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfb30e1-01b1-4c3e-92e2-d5edf10d2eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:28.345532 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb30e1-01b1-4c3e-92e2-d5edf10d2eec None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:28.345807 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb30e1-01b1-4c3e-92e2-d5edf10d2eec None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:28.346075 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb30e1-01b1-4c3e-92e2-d5edf10d2eec None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:28.353085 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfb30e1-01b1-4c3e-92e2-d5edf10d2eec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:28.354509 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfb30e1-01b1-4c3e-92e2-d5edf10d2eec None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:28.357323 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:28.367226 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-ccfb30e1-01b1-4c3e-92e2-d5edf10d2eec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:24:28.367226 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:28.367226 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:28.367226 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:28.367226 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:28.367226 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:28.367226 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:28.367226 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:28.367226 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:28.367226 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:28.367226 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:28.367226 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:28.367226 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 09:24:28.367226 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 09:24:28.367226 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 09:24:28.368107 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 09:24:28.368107 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:28.368107 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:28.368107 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:28.368107 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:28.368107 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:28.368107 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:28.368107 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:28.368107 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:28.368107 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:28.368107 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:28.368107 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:28.368107 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:28.368107 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:28.368107 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:28.368107 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:28.368107 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:28.368989 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 09:24:28.368989 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 09:24:28.368989 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 09:24:28.368989 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 09:24:28.368989 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 09:24:28.368989 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 09:24:28.368989 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:24:28.368989 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:28.368989 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 229/452] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Oct 27 09:24:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:28.369437 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfb30e1-01b1-4c3e-92e2-d5edf10d2eec None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:28.404416 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb853a-2a8c-4bba-8631-2076546a588a None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:28.437416 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01da926f-410c-4b1a-9fa4-cec15c5b685e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:28.438030 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da926f-410c-4b1a-9fa4-cec15c5b685e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:28.438304 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da926f-410c-4b1a-9fa4-cec15c5b685e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:28.438572 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da926f-410c-4b1a-9fa4-cec15c5b685e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:28.448582 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01da926f-410c-4b1a-9fa4-cec15c5b685e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:24:28.453713 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-01da926f-410c-4b1a-9fa4-cec15c5b685e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 27 09:24:28.453713 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:28.453713 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:28.453713 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:28.453713 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:28.453713 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:28.453713 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:28.453713 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:28.453713 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:28.453713 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:28.453713 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:28.453713 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:28.453713 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:24:28.453713 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:24:28.453713 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:24:28.453713 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:24:28.454764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:24:28.454764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:24:28.454764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:24:28.454764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:24:28.454764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:28.454764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:28.454764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:28.454764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:28.454764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:28.454764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:28.454764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:28.454764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:28.454764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:28.454764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:28.454764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:28.454764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:28.457553 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:28.457553 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:28.457553 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:28.457553 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:28.457553 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:24:28.457553 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:24:28.457553 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:24:28.457553 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:24:28.457553 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:24:28.457553 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:24:28.457553 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 27 09:24:28.457553 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:28.457553 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 224/453] 10.222.0.70 () {56 vars in 1102 bytes} [Thu Oct 27 09:24:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:28.457553 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01da926f-410c-4b1a-9fa4-cec15c5b685e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:28.470415 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfb30e1-01b1-4c3e-92e2-d5edf10d2eec None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:28.503783 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260a88ac-66ca-4c6a-812f-6543069204fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:28.504434 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a88ac-66ca-4c6a-812f-6543069204fb None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:28.504689 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a88ac-66ca-4c6a-812f-6543069204fb None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:28.504963 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a88ac-66ca-4c6a-812f-6543069204fb None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:28.505589 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260a88ac-66ca-4c6a-812f-6543069204fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:28.505892 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260a88ac-66ca-4c6a-812f-6543069204fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:28.507781 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260a88ac-66ca-4c6a-812f-6543069204fb None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:28.519212 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 230/454] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:24:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:28.520059 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260a88ac-66ca-4c6a-812f-6543069204fb None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:28.544605 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:28.554471 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01da926f-410c-4b1a-9fa4-cec15c5b685e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:28.579526 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:28.588834 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2bb4e8-ff30-40c2-b32b-338d876f7bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:28.589428 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2bb4e8-ff30-40c2-b32b-338d876f7bef None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:28.589698 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2bb4e8-ff30-40c2-b32b-338d876f7bef None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:28.589961 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2bb4e8-ff30-40c2-b32b-338d876f7bef None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:28.590665 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2bb4e8-ff30-40c2-b32b-338d876f7bef None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:28.590961 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2bb4e8-ff30-40c2-b32b-338d876f7bef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:28.592398 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2bb4e8-ff30-40c2-b32b-338d876f7bef None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:28.609227 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 225/455] 10.222.0.70 () {56 vars in 1137 bytes} [Thu Oct 27 09:24:28 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:28.610006 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2bb4e8-ff30-40c2-b32b-338d876f7bef None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:28.621329 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260a88ac-66ca-4c6a-812f-6543069204fb None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:28.655396 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93655a2d-9bbb-4c50-bd27-5c25cc67562c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:28.656015 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93655a2d-9bbb-4c50-bd27-5c25cc67562c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:28.656293 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93655a2d-9bbb-4c50-bd27-5c25cc67562c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:28.656586 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93655a2d-9bbb-4c50-bd27-5c25cc67562c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:28.666370 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93655a2d-9bbb-4c50-bd27-5c25cc67562c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:28.667808 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93655a2d-9bbb-4c50-bd27-5c25cc67562c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:28.682857 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-93655a2d-9bbb-4c50-bd27-5c25cc67562c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:28.682857 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:28.682857 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:28.682857 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:28.682857 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:28.682857 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:28.682857 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:28.682857 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:28.682857 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:28.682857 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:28.682857 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:28.682857 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:28.682857 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:24:28.682857 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:24:28.682857 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:24:28.682857 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:24:28.683911 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:28.683911 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:28.683911 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:28.683911 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:28.683911 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:28.683911 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:28.683911 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:28.683911 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:28.683911 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:28.683911 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:28.683911 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:28.683911 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:28.683911 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:28.683911 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:28.683911 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:28.683911 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:28.683911 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:24:28.685090 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:24:28.685090 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:28.685090 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:28.685090 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 231/456] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:24:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:28.685090 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93655a2d-9bbb-4c50-bd27-5c25cc67562c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:28.713502 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2bb4e8-ff30-40c2-b32b-338d876f7bef None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:28.746758 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce69327-ab92-48df-88f5-00e8539c54ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:28.747355 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce69327-ab92-48df-88f5-00e8539c54ab None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:28.747633 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce69327-ab92-48df-88f5-00e8539c54ab None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:28.747896 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce69327-ab92-48df-88f5-00e8539c54ab None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:28.748600 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce69327-ab92-48df-88f5-00e8539c54ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:28.748899 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce69327-ab92-48df-88f5-00e8539c54ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:28.750325 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce69327-ab92-48df-88f5-00e8539c54ab None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:28.761785 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 226/457] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Oct 27 09:24:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:28.762585 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce69327-ab92-48df-88f5-00e8539c54ab None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:28.773697 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:28.783721 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93655a2d-9bbb-4c50-bd27-5c25cc67562c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:28.796696 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:28.817403 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb5e751-f6e3-4871-bfa9-939d90faab07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:28.818027 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb5e751-f6e3-4871-bfa9-939d90faab07 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:28.818281 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb5e751-f6e3-4871-bfa9-939d90faab07 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:28.818550 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb5e751-f6e3-4871-bfa9-939d90faab07 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:28.819187 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb5e751-f6e3-4871-bfa9-939d90faab07 None admin] RBAC: Adding query filter params (name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:28.819509 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb5e751-f6e3-4871-bfa9-939d90faab07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:28.820894 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb5e751-f6e3-4871-bfa9-939d90faab07 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:28.828557 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 232/458] 10.222.0.70 () {56 vars in 1103 bytes} [Thu Oct 27 09:24:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:28.829410 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb5e751-f6e3-4871-bfa9-939d90faab07 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:28.864058 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce69327-ab92-48df-88f5-00e8539c54ab None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:28.896898 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9164b6-226d-4619-a94a-ac5259c10e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:28.898118 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9164b6-226d-4619-a94a-ac5259c10e92 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:28.898395 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9164b6-226d-4619-a94a-ac5259c10e92 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:28.898660 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9164b6-226d-4619-a94a-ac5259c10e92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:28.905182 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9164b6-226d-4619-a94a-ac5259c10e92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:28.906625 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9164b6-226d-4619-a94a-ac5259c10e92 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:28.918130 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-ad9164b6-226d-4619-a94a-ac5259c10e92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:24:28.918130 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:28.918130 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:28.918130 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:28.918130 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:28.918130 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:28.918130 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:28.918130 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:28.918130 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:28.918130 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:28.918130 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:28.918130 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:28.918130 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 09:24:28.918130 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 09:24:28.918130 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 09:24:28.918982 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 09:24:28.918982 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:28.918982 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:28.918982 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:28.918982 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:28.918982 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:28.918982 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:28.918982 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:28.918982 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:28.918982 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:28.918982 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:28.918982 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:28.918982 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:28.918982 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:28.918982 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:28.918982 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:28.918982 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:28.919814 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 09:24:28.919814 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 09:24:28.919814 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 09:24:28.919814 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 09:24:28.919814 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 09:24:28.919814 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 09:24:28.919814 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:24:28.919814 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:28.919814 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 227/459] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Oct 27 09:24:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:28.920291 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9164b6-226d-4619-a94a-ac5259c10e92 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:28.933293 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb5e751-f6e3-4871-bfa9-939d90faab07 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:28.970886 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf1d8c1-1e8e-43bf-b4eb-68d55234cfcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:28.971668 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf1d8c1-1e8e-43bf-b4eb-68d55234cfcb None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:28.972095 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf1d8c1-1e8e-43bf-b4eb-68d55234cfcb None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:28.972538 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf1d8c1-1e8e-43bf-b4eb-68d55234cfcb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:28.987285 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:29.017017 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:29.020708 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9164b6-226d-4619-a94a-ac5259c10e92 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:29.043318 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf1d8c1-1e8e-43bf-b4eb-68d55234cfcb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:29.044825 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf1d8c1-1e8e-43bf-b4eb-68d55234cfcb None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:29.053478 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5314c75-d42e-498e-a84f-58f786f22315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:29.054101 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5314c75-d42e-498e-a84f-58f786f22315 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:29.054380 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5314c75-d42e-498e-a84f-58f786f22315 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:29.054648 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5314c75-d42e-498e-a84f-58f786f22315 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:29.055274 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5314c75-d42e-498e-a84f-58f786f22315 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:29.055572 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5314c75-d42e-498e-a84f-58f786f22315 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:29.056950 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5314c75-d42e-498e-a84f-58f786f22315 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:29.068337 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 228/460] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:24:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:29.069145 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5314c75-d42e-498e-a84f-58f786f22315 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:29.144406 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 233/461] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Oct 27 09:24:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 27 09:24:29.145092 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf1d8c1-1e8e-43bf-b4eb-68d55234cfcb None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:29.173753 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5314c75-d42e-498e-a84f-58f786f22315 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:29.199879 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:29.207732 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820cab98-172b-43ed-9682-5ac134b5dea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:29.208324 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820cab98-172b-43ed-9682-5ac134b5dea4 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:29.208629 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820cab98-172b-43ed-9682-5ac134b5dea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:29.208901 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820cab98-172b-43ed-9682-5ac134b5dea4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:29.209603 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820cab98-172b-43ed-9682-5ac134b5dea4 None admin] RBAC: Adding query filter params (role.id=5146b19817b54113be3f81111ef9d787, scope.project.id=16ac205ec6f241d69d5e6b8408f09f38, user.id=dbe4a6e4b6184826b30ac4fc13ad9b1f) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:29.209906 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820cab98-172b-43ed-9682-5ac134b5dea4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:29.211908 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820cab98-172b-43ed-9682-5ac134b5dea4 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:29.217647 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 229/462] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Oct 27 09:24:29 2022] GET /identity/v3/role_assignments?user.id=dbe4a6e4b6184826b30ac4fc13ad9b1f&scope.project.id=16ac205ec6f241d69d5e6b8408f09f38&role.id=5146b19817b54113be3f81111ef9d787 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:29.218065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:29.218253 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820cab98-172b-43ed-9682-5ac134b5dea4 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:29.245618 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf1d8c1-1e8e-43bf-b4eb-68d55234cfcb None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:29.293353 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef8d7a-061b-4231-b75c-7e0a483db9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:29.294085 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef8d7a-061b-4231-b75c-7e0a483db9bf None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:29.294424 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef8d7a-061b-4231-b75c-7e0a483db9bf None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:29.294756 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef8d7a-061b-4231-b75c-7e0a483db9bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:29.308877 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ef8d7a-061b-4231-b75c-7e0a483db9bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:29.310484 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ef8d7a-061b-4231-b75c-7e0a483db9bf None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:29.327492 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-03ef8d7a-061b-4231-b75c-7e0a483db9bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:29.327492 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:29.327492 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:29.327492 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:29.327492 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:29.327492 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:29.327492 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:29.327492 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:29.327492 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:29.327492 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:29.327492 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:29.327492 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:29.327492 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:24:29.327492 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:24:29.327492 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:24:29.327492 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:24:29.328652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:29.328652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:29.328652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:29.328652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:29.328652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:29.328652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:29.328652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:29.328652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:29.328652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:29.328652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:29.328652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:29.328652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:29.328652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:29.328652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:29.328652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:29.328652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:29.328652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:24:29.329737 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:24:29.329737 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:29.329737 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:29.329898 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 234/463] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:24:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:29.330602 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef8d7a-061b-4231-b75c-7e0a483db9bf None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:29.340355 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820cab98-172b-43ed-9682-5ac134b5dea4 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:29.379500 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a30be4-0d5d-4ba6-9e51-34fbb80dcb49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:29.380428 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a30be4-0d5d-4ba6-9e51-34fbb80dcb49 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:29.380801 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a30be4-0d5d-4ba6-9e51-34fbb80dcb49 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:29.381142 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a30be4-0d5d-4ba6-9e51-34fbb80dcb49 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:29.381837 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a30be4-0d5d-4ba6-9e51-34fbb80dcb49 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:29.382173 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a30be4-0d5d-4ba6-9e51-34fbb80dcb49 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:29.383622 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a30be4-0d5d-4ba6-9e51-34fbb80dcb49 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:29.402549 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 230/464] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Oct 27 09:24:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:29.404669 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a30be4-0d5d-4ba6-9e51-34fbb80dcb49 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:29.446881 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef8d7a-061b-4231-b75c-7e0a483db9bf None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:29.468133 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:29.474447 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:29.484787 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c9e828-e166-41b8-ba0b-4d7b4c5702aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:29.485362 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9e828-e166-41b8-ba0b-4d7b4c5702aa None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:29.485640 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9e828-e166-41b8-ba0b-4d7b4c5702aa None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:29.485913 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9e828-e166-41b8-ba0b-4d7b4c5702aa None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:29.495647 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c9e828-e166-41b8-ba0b-4d7b4c5702aa None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:29.497483 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c9e828-e166-41b8-ba0b-4d7b4c5702aa None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:29.510422 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-67c9e828-e166-41b8-ba0b-4d7b4c5702aa None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 27 09:24:29.510422 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:29.510422 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:29.510422 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:29.510422 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:29.510422 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:29.510422 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:29.510422 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:29.510422 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:29.510422 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:29.510422 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:29.510422 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:29.510422 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 09:24:29.510422 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 09:24:29.510422 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 09:24:29.510422 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 09:24:29.511334 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:29.511334 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:29.511334 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 09:24:29.511334 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:24:29.511334 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 09:24:29.511334 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:24:29.511334 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:29.511334 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:29.511334 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:29.511334 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:29.511334 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:29.511334 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:29.511334 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:29.511334 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:29.511334 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:29.511334 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:29.511334 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:29.511334 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:29.512204 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:29.512204 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:29.512204 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 09:24:29.512204 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 09:24:29.512204 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 09:24:29.512204 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 09:24:29.512204 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 09:24:29.512204 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 09:24:29.512204 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 27 09:24:29.512204 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:29.512204 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 235/465] 10.222.0.70 () {56 vars in 1102 bytes} [Thu Oct 27 09:24:29 2022] GET /identity/v3/users/neutron => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:29.512790 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c9e828-e166-41b8-ba0b-4d7b4c5702aa None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:29.524583 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a30be4-0d5d-4ba6-9e51-34fbb80dcb49 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:29.559259 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00651d5-7ad9-4faa-b583-df0936eb6f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:29.559900 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00651d5-7ad9-4faa-b583-df0936eb6f20 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:29.560060 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00651d5-7ad9-4faa-b583-df0936eb6f20 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:29.560325 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00651d5-7ad9-4faa-b583-df0936eb6f20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:29.570904 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00651d5-7ad9-4faa-b583-df0936eb6f20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:29.572439 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00651d5-7ad9-4faa-b583-df0936eb6f20 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:29.587806 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-a00651d5-7ad9-4faa-b583-df0936eb6f20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:29.587806 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:29.587806 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:29.587806 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:29.587806 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:29.587806 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:29.587806 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:29.587806 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:29.587806 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:29.587806 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:29.587806 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:29.587806 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:29.587806 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:24:29.587806 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:24:29.587806 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:24:29.587806 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:24:29.588738 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:29.588738 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:29.588738 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:29.588738 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:29.588738 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:29.588738 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:29.588738 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:29.588738 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:29.588738 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:29.588738 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:29.588738 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:29.588738 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:29.588738 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:29.588738 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:29.588738 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:29.588738 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:29.588738 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:24:29.589578 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:24:29.589578 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:29.589578 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:29.589578 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 231/466] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:24:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:29.590471 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00651d5-7ad9-4faa-b583-df0936eb6f20 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:29.630692 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c9e828-e166-41b8-ba0b-4d7b4c5702aa None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:29.668636 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef980fd8-3927-40f9-8378-995b3f16f2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:29.669290 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef980fd8-3927-40f9-8378-995b3f16f2b5 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:29.669500 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef980fd8-3927-40f9-8378-995b3f16f2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:29.669758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef980fd8-3927-40f9-8378-995b3f16f2b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:29.670381 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef980fd8-3927-40f9-8378-995b3f16f2b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:29.670681 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef980fd8-3927-40f9-8378-995b3f16f2b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:29.672029 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef980fd8-3927-40f9-8378-995b3f16f2b5 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:29.683915 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 236/467] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:24:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:29.684557 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef980fd8-3927-40f9-8378-995b3f16f2b5 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:29.699834 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00651d5-7ad9-4faa-b583-df0936eb6f20 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:29.702321 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:29.741603 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:29.742081 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f804c9-705a-4619-89c7-2aea552dee6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:29.743133 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f804c9-705a-4619-89c7-2aea552dee6b None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:29.743133 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f804c9-705a-4619-89c7-2aea552dee6b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:29.743337 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f804c9-705a-4619-89c7-2aea552dee6b None admin] PATH_INFO: `/v3/projects/16ac205ec6f241d69d5e6b8408f09f38/users/eecf79ea4700496c89b1a67c614b34d0/roles/5146b19817b54113be3f81111ef9d787` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:29.758910 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f804c9-705a-4619-89c7-2aea552dee6b None admin] RBAC: Authorizing `identity:create_grant(project_id=16ac205ec6f241d69d5e6b8408f09f38, user_id=eecf79ea4700496c89b1a67c614b34d0, role_id=5146b19817b54113be3f81111ef9d787)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:29.760360 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f804c9-705a-4619-89c7-2aea552dee6b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:29.784920 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 232/468] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Oct 27 09:24:29 2022] PUT /identity/v3/projects/16ac205ec6f241d69d5e6b8408f09f38/users/eecf79ea4700496c89b1a67c614b34d0/roles/5146b19817b54113be3f81111ef9d787 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:24:29.784920 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f804c9-705a-4619-89c7-2aea552dee6b None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:29.811674 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef980fd8-3927-40f9-8378-995b3f16f2b5 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:29.858493 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21b7315-3fd7-4e34-9667-fe763927e876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:29.859212 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21b7315-3fd7-4e34-9667-fe763927e876 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:29.859491 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21b7315-3fd7-4e34-9667-fe763927e876 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:29.859758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21b7315-3fd7-4e34-9667-fe763927e876 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:29.860643 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21b7315-3fd7-4e34-9667-fe763927e876 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:29.860998 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21b7315-3fd7-4e34-9667-fe763927e876 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:29.862509 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21b7315-3fd7-4e34-9667-fe763927e876 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:29.886773 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 237/469] 10.222.0.70 () {56 vars in 1139 bytes} [Thu Oct 27 09:24:29 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:29.887565 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21b7315-3fd7-4e34-9667-fe763927e876 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:29.900820 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f804c9-705a-4619-89c7-2aea552dee6b None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:29.942086 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8090c201-165a-465b-9d04-f75164797e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:29.942792 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8090c201-165a-465b-9d04-f75164797e4f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:29.943074 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8090c201-165a-465b-9d04-f75164797e4f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:29.943346 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8090c201-165a-465b-9d04-f75164797e4f None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:29.943996 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8090c201-165a-465b-9d04-f75164797e4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:29.945045 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8090c201-165a-465b-9d04-f75164797e4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:29.946648 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8090c201-165a-465b-9d04-f75164797e4f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:29.964666 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 233/470] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:24:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:29.965594 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8090c201-165a-465b-9d04-f75164797e4f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:29.983713 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:30.026092 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21b7315-3fd7-4e34-9667-fe763927e876 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:30.029806 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:30.073728 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2768a446-361d-440a-8971-3bc176e02243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:30.074259 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768a446-361d-440a-8971-3bc176e02243 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:30.075071 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768a446-361d-440a-8971-3bc176e02243 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:30.075358 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768a446-361d-440a-8971-3bc176e02243 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:30.086446 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2768a446-361d-440a-8971-3bc176e02243 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:30.088182 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2768a446-361d-440a-8971-3bc176e02243 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:30.089468 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8090c201-165a-465b-9d04-f75164797e4f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:30.103614 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-2768a446-361d-440a-8971-3bc176e02243 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 27 09:24:30.103614 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:30.103614 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:30.103614 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:30.103614 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:30.103614 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:30.103614 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:30.103614 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:30.103614 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:30.103614 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:30.103614 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:30.103614 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:30.103614 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 09:24:30.103614 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 09:24:30.103614 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 09:24:30.103614 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 09:24:30.104826 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:30.104826 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:30.104826 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 09:24:30.104826 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:24:30.104826 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 09:24:30.104826 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:24:30.104826 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:30.104826 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:30.104826 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:30.104826 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:30.104826 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:30.104826 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:30.104826 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:30.104826 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:30.104826 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:30.104826 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:30.104826 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:30.104826 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:30.106101 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:30.106101 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:30.106101 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 09:24:30.106101 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 09:24:30.106101 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 09:24:30.106101 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 09:24:30.106101 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 09:24:30.106101 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 09:24:30.106101 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 27 09:24:30.106101 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:30.108596 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 238/471] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Oct 27 09:24:29 2022] GET /identity/v3/users/glance => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:30.110558 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2768a446-361d-440a-8971-3bc176e02243 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:30.135815 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36031b0e-3963-42bc-8165-bd6ca5b03fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:30.136930 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36031b0e-3963-42bc-8165-bd6ca5b03fce None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:30.137537 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36031b0e-3963-42bc-8165-bd6ca5b03fce None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:30.137988 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36031b0e-3963-42bc-8165-bd6ca5b03fce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:30.215808 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36031b0e-3963-42bc-8165-bd6ca5b03fce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:30.217774 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36031b0e-3963-42bc-8165-bd6ca5b03fce None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:30.234440 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:30.246566 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2768a446-361d-440a-8971-3bc176e02243 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:30.291771 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e99cca3-c4e8-438c-b2f7-586e2e540a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:30.292408 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e99cca3-c4e8-438c-b2f7-586e2e540a5e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:30.292720 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e99cca3-c4e8-438c-b2f7-586e2e540a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:30.292994 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e99cca3-c4e8-438c-b2f7-586e2e540a5e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:30.302615 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e99cca3-c4e8-438c-b2f7-586e2e540a5e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:30.304346 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e99cca3-c4e8-438c-b2f7-586e2e540a5e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:30.320647 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-4e99cca3-c4e8-438c-b2f7-586e2e540a5e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:24:30.320647 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:30.320647 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:30.320647 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:30.320647 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:30.320647 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:30.320647 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:30.320647 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:30.320647 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:30.320647 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:30.320647 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:30.320647 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:30.320647 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 09:24:30.320647 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 09:24:30.320647 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 09:24:30.321509 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 09:24:30.321509 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:30.321509 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:30.321509 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:30.321509 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:30.321509 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:30.321509 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:30.321509 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:30.321509 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:30.321509 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:30.321509 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:30.321509 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:30.321509 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:30.321509 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:30.321509 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:30.321509 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:30.321509 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:30.322342 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 09:24:30.322342 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 09:24:30.322342 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 09:24:30.322342 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 09:24:30.322342 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 09:24:30.322342 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 09:24:30.322342 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:24:30.322342 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:30.330333 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 239/472] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Oct 27 09:24:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:30.331463 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e99cca3-c4e8-438c-b2f7-586e2e540a5e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:30.342439 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 234/473] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Oct 27 09:24:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 27 09:24:30.342439 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36031b0e-3963-42bc-8165-bd6ca5b03fce None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:30.363146 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:30.453381 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36031b0e-3963-42bc-8165-bd6ca5b03fce None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:30.462868 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e99cca3-c4e8-438c-b2f7-586e2e540a5e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:30.468971 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:30.492426 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5584d06-d91e-4201-ab81-675f9e9bb102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:30.493457 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5584d06-d91e-4201-ab81-675f9e9bb102 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:30.493457 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5584d06-d91e-4201-ab81-675f9e9bb102 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:30.493674 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5584d06-d91e-4201-ab81-675f9e9bb102 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:30.494350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5584d06-d91e-4201-ab81-675f9e9bb102 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:30.494656 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5584d06-d91e-4201-ab81-675f9e9bb102 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:30.496086 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5584d06-d91e-4201-ab81-675f9e9bb102 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:30.509067 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 235/474] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Oct 27 09:24:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:30.509901 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5584d06-d91e-4201-ab81-675f9e9bb102 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:30.510808 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fb757c-f8ec-4554-93d3-70bfb286963c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:30.511646 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fb757c-f8ec-4554-93d3-70bfb286963c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:30.512101 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fb757c-f8ec-4554-93d3-70bfb286963c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:30.512869 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fb757c-f8ec-4554-93d3-70bfb286963c None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:30.514731 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fb757c-f8ec-4554-93d3-70bfb286963c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:30.515225 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fb757c-f8ec-4554-93d3-70bfb286963c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:30.516892 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fb757c-f8ec-4554-93d3-70bfb286963c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:30.546549 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 240/475] 10.222.0.70 () {56 vars in 1137 bytes} [Thu Oct 27 09:24:30 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:30.546549 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fb757c-f8ec-4554-93d3-70bfb286963c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:30.627976 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5584d06-d91e-4201-ab81-675f9e9bb102 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:30.664224 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:30.665771 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25671d57-4276-4fdc-ab34-4dd0ce212823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:30.666583 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25671d57-4276-4fdc-ab34-4dd0ce212823 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:30.667003 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25671d57-4276-4fdc-ab34-4dd0ce212823 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:30.667455 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25671d57-4276-4fdc-ab34-4dd0ce212823 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:30.680631 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25671d57-4276-4fdc-ab34-4dd0ce212823 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:30.682436 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25671d57-4276-4fdc-ab34-4dd0ce212823 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:30.686948 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fb757c-f8ec-4554-93d3-70bfb286963c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:30.698586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-25671d57-4276-4fdc-ab34-4dd0ce212823 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:30.698586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:30.698586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:30.698586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:30.698586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:30.698586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:30.698586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:30.698586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:30.698586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:30.698586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:30.698586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:30.698586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:30.698586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:24:30.698586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:24:30.698586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:24:30.698586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:24:30.699514 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:30.699514 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:30.699514 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:30.699514 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:30.699514 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:30.699514 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:30.699514 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:30.699514 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:30.699514 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:30.699514 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:30.699514 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:30.699514 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:30.699514 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:30.699514 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:30.699514 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:30.699514 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:30.699514 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:24:30.700330 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:24:30.700330 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:30.700330 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:30.702561 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 236/476] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:24:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:30.703625 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25671d57-4276-4fdc-ab34-4dd0ce212823 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:30.728185 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1321a7f9-e73c-46a3-9e2b-d9fa8bd56e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:30.729119 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1321a7f9-e73c-46a3-9e2b-d9fa8bd56e1a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:30.729602 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1321a7f9-e73c-46a3-9e2b-d9fa8bd56e1a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:30.730093 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1321a7f9-e73c-46a3-9e2b-d9fa8bd56e1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:30.730999 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1321a7f9-e73c-46a3-9e2b-d9fa8bd56e1a None admin] RBAC: Adding query filter params (role.id=5146b19817b54113be3f81111ef9d787, scope.project.id=16ac205ec6f241d69d5e6b8408f09f38, user.id=5b8e13ce967449c1b12783183e953579) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:30.731510 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1321a7f9-e73c-46a3-9e2b-d9fa8bd56e1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:30.733203 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1321a7f9-e73c-46a3-9e2b-d9fa8bd56e1a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:30.739868 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 241/477] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Oct 27 09:24:30 2022] GET /identity/v3/role_assignments?user.id=5b8e13ce967449c1b12783183e953579&scope.project.id=16ac205ec6f241d69d5e6b8408f09f38&role.id=5146b19817b54113be3f81111ef9d787 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:30.741027 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1321a7f9-e73c-46a3-9e2b-d9fa8bd56e1a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:30.741845 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:30.845395 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25671d57-4276-4fdc-ab34-4dd0ce212823 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:30.854184 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1321a7f9-e73c-46a3-9e2b-d9fa8bd56e1a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:30.895431 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c214da-0a4c-48ae-b781-4cfd555f35f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:30.897403 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c214da-0a4c-48ae-b781-4cfd555f35f7 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:30.898970 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c214da-0a4c-48ae-b781-4cfd555f35f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:30.899830 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636a4ea-4614-45b1-9f37-af40e2e9e2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:30.900344 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c214da-0a4c-48ae-b781-4cfd555f35f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:30.901253 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c214da-0a4c-48ae-b781-4cfd555f35f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:30.904850 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636a4ea-4614-45b1-9f37-af40e2e9e2cd None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:30.904850 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636a4ea-4614-45b1-9f37-af40e2e9e2cd None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:30.904850 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636a4ea-4614-45b1-9f37-af40e2e9e2cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:30.905097 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c214da-0a4c-48ae-b781-4cfd555f35f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:30.905327 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c214da-0a4c-48ae-b781-4cfd555f35f7 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:30.936073 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 242/478] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:24:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:30.941700 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c214da-0a4c-48ae-b781-4cfd555f35f7 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:30.954680 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:30.997230 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1636a4ea-4614-45b1-9f37-af40e2e9e2cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:30.999440 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1636a4ea-4614-45b1-9f37-af40e2e9e2cd None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:31.034561 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:31.083416 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c214da-0a4c-48ae-b781-4cfd555f35f7 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:31.124192 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fd3dde-6875-4122-ab58-0bfa550d9d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:31.125284 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd3dde-6875-4122-ab58-0bfa550d9d7d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:31.125872 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd3dde-6875-4122-ab58-0bfa550d9d7d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:31.126363 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd3dde-6875-4122-ab58-0bfa550d9d7d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:31.133820 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 237/479] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Oct 27 09:24:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 27 09:24:31.136561 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fd3dde-6875-4122-ab58-0bfa550d9d7d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:31.136561 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fd3dde-6875-4122-ab58-0bfa550d9d7d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:31.139643 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636a4ea-4614-45b1-9f37-af40e2e9e2cd None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:31.151643 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-a6fd3dde-6875-4122-ab58-0bfa550d9d7d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:24:31.151643 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:31.151643 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:31.151643 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:31.151643 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:31.151643 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:31.151643 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:31.151643 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:31.151643 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:31.151643 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:31.151643 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:31.151643 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:31.151643 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 09:24:31.151643 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 09:24:31.151643 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 09:24:31.152507 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 09:24:31.152507 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:31.152507 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:31.152507 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:31.152507 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:31.152507 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:31.152507 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:31.152507 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:31.152507 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:31.152507 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:31.152507 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:31.152507 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:31.152507 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:31.152507 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:31.152507 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:31.152507 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:31.152507 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:31.153527 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 09:24:31.153527 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 09:24:31.153527 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 09:24:31.153527 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 09:24:31.153527 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 09:24:31.153527 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 09:24:31.153527 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:24:31.153527 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:31.160432 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 243/480] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Oct 27 09:24:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 09:24:31.161918 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fd3dde-6875-4122-ab58-0bfa550d9d7d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:31.240878 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:31.271702 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636a4ea-4614-45b1-9f37-af40e2e9e2cd None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:31.288622 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fd3dde-6875-4122-ab58-0bfa550d9d7d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:31.302961 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:31.325221 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b06e491-95c2-432d-990e-8c61f4fd9ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:31.325908 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b06e491-95c2-432d-990e-8c61f4fd9ba5 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:31.326250 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b06e491-95c2-432d-990e-8c61f4fd9ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:31.326525 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b06e491-95c2-432d-990e-8c61f4fd9ba5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:31.347192 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b06e491-95c2-432d-990e-8c61f4fd9ba5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:31.348679 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b06e491-95c2-432d-990e-8c61f4fd9ba5 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:31.361670 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d486b-97ad-47be-b779-4acaef64fe4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:31.362281 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d486b-97ad-47be-b779-4acaef64fe4b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:31.362560 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d486b-97ad-47be-b779-4acaef64fe4b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:31.362834 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d486b-97ad-47be-b779-4acaef64fe4b None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:31.363572 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d486b-97ad-47be-b779-4acaef64fe4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:31.363876 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d486b-97ad-47be-b779-4acaef64fe4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:31.365480 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d486b-97ad-47be-b779-4acaef64fe4b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:31.374763 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-2b06e491-95c2-432d-990e-8c61f4fd9ba5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:31.374763 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:31.374763 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:31.374763 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:31.374763 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:31.374763 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:31.374763 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:31.374763 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:31.374763 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:31.374763 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:31.374763 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:31.374763 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:31.374763 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:24:31.374763 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:24:31.374763 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:24:31.374763 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:24:31.375667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:31.375667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:31.375667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:31.375667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:31.375667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:31.375667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:31.375667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:31.375667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:31.375667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:31.375667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:31.375667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:31.375667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:31.375667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:31.375667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:31.375667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:31.375667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:31.375667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:24:31.376606 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:24:31.376606 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:31.376606 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:31.380504 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 238/481] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:24:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 09:24:31.380504 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b06e491-95c2-432d-990e-8c61f4fd9ba5 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:31.399672 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 244/482] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Oct 27 09:24:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:31.408723 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d486b-97ad-47be-b779-4acaef64fe4b None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:31.573068 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b06e491-95c2-432d-990e-8c61f4fd9ba5 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:31.576073 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d486b-97ad-47be-b779-4acaef64fe4b None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:31.613125 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:31.621996 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0cdba5-b291-48ed-9ecb-971cc17a8de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:31.622787 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0cdba5-b291-48ed-9ecb-971cc17a8de1 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:31.623168 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f1509-70d7-4082-9e2c-25d673c40ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:31.623403 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0cdba5-b291-48ed-9ecb-971cc17a8de1 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:31.623770 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f1509-70d7-4082-9e2c-25d673c40ff2 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:31.623966 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0cdba5-b291-48ed-9ecb-971cc17a8de1 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:31.624068 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f1509-70d7-4082-9e2c-25d673c40ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:31.624293 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f1509-70d7-4082-9e2c-25d673c40ff2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:31.625001 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0cdba5-b291-48ed-9ecb-971cc17a8de1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:31.625108 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894f1509-70d7-4082-9e2c-25d673c40ff2 None admin] RBAC: Adding query filter params (role.id=5146b19817b54113be3f81111ef9d787, scope.project.id=16ac205ec6f241d69d5e6b8408f09f38, user.id=45d8920d9a7940abb112dbbb3d7f6e65) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:31.625325 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894f1509-70d7-4082-9e2c-25d673c40ff2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:31.625666 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0cdba5-b291-48ed-9ecb-971cc17a8de1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:31.626741 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894f1509-70d7-4082-9e2c-25d673c40ff2 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:31.627418 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0cdba5-b291-48ed-9ecb-971cc17a8de1 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:31.633726 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 245/483] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Oct 27 09:24:31 2022] GET /identity/v3/role_assignments?user.id=45d8920d9a7940abb112dbbb3d7f6e65&scope.project.id=16ac205ec6f241d69d5e6b8408f09f38&role.id=5146b19817b54113be3f81111ef9d787 => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:24:31.634976 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6059d2a-2743-4150-a097-7f0129fc7cc4 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:31.635260 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6059d2a-2743-4150-a097-7f0129fc7cc4 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:31.635529 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6059d2a-2743-4150-a097-7f0129fc7cc4 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:31.636957 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 246/484] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:31.641800 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 239/485] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:24:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:24:31.642688 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b4d52-527e-4f1c-940e-006596e9e072 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:31.642975 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b4d52-527e-4f1c-940e-006596e9e072 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:31.643244 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b4d52-527e-4f1c-940e-006596e9e072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:31.649783 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0cdba5-b291-48ed-9ecb-971cc17a8de1 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:31.665795 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-817b4d52-527e-4f1c-940e-006596e9e072 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:31.666941 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-817b4d52-527e-4f1c-940e-006596e9e072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:31.710646 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:31.805056 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0cdba5-b291-48ed-9ecb-971cc17a8de1 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:31.821260 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 247/486] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 27 09:24:31.865061 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d32626-438a-400f-b05e-71f82e770e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:31.865990 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d32626-438a-400f-b05e-71f82e770e04 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:31.866979 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d32626-438a-400f-b05e-71f82e770e04 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:31.867421 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d32626-438a-400f-b05e-71f82e770e04 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:31.875361 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d32626-438a-400f-b05e-71f82e770e04 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:31.877090 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d32626-438a-400f-b05e-71f82e770e04 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:31.889333 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-72d32626-438a-400f-b05e-71f82e770e04 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:24:31.889333 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:31.889333 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:31.889333 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:31.889333 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:31.889333 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:31.889333 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:31.889333 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:31.889333 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:31.889333 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:31.889333 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:31.889333 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:31.889333 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 09:24:31.889333 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 09:24:31.889333 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 09:24:31.890195 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 09:24:31.890195 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:31.890195 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:31.890195 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:31.890195 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:31.890195 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:31.890195 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:31.890195 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:31.890195 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:31.890195 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:31.890195 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:31.890195 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:31.890195 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:31.890195 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:31.890195 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:31.890195 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:31.890195 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:31.891020 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 09:24:31.891020 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 09:24:31.891020 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 09:24:31.891020 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 09:24:31.891020 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 09:24:31.891020 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 09:24:31.891020 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:24:31.891020 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:31.893046 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 240/487] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Oct 27 09:24:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:31.901597 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817b4d52-527e-4f1c-940e-006596e9e072 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:32.007096 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817b4d52-527e-4f1c-940e-006596e9e072 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:32.032420 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:32.049367 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c481cb2-d671-4d36-9397-d9c5c2a2070a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:32.050327 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c481cb2-d671-4d36-9397-d9c5c2a2070a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:32.050829 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c481cb2-d671-4d36-9397-d9c5c2a2070a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:32.051300 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c481cb2-d671-4d36-9397-d9c5c2a2070a None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:32.052184 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c481cb2-d671-4d36-9397-d9c5c2a2070a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:32.052734 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c481cb2-d671-4d36-9397-d9c5c2a2070a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:32.054924 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c481cb2-d671-4d36-9397-d9c5c2a2070a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:32.072619 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 248/488] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Oct 27 09:24:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:32.077600 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d32626-438a-400f-b05e-71f82e770e04 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:32.161268 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c481cb2-d671-4d36-9397-d9c5c2a2070a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:32.205153 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:32.208996 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d32626-438a-400f-b05e-71f82e770e04 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:32.264241 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda92e3-6b4b-476e-b553-80aa0c188bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:32.264802 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda92e3-6b4b-476e-b553-80aa0c188bd1 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:32.265159 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda92e3-6b4b-476e-b553-80aa0c188bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:32.265354 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda92e3-6b4b-476e-b553-80aa0c188bd1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:32.266128 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efda92e3-6b4b-476e-b553-80aa0c188bd1 None admin] RBAC: Adding query filter params (role.id=5146b19817b54113be3f81111ef9d787, scope.project.id=16ac205ec6f241d69d5e6b8408f09f38, user.id=72b0538f0e42446887b1d95b2fb41fb2) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:32.266439 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efda92e3-6b4b-476e-b553-80aa0c188bd1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:32.268194 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efda92e3-6b4b-476e-b553-80aa0c188bd1 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:32.274413 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c481cb2-d671-4d36-9397-d9c5c2a2070a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:32.279528 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 241/489] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Oct 27 09:24:32 2022] GET /identity/v3/role_assignments?user.id=72b0538f0e42446887b1d95b2fb41fb2&scope.project.id=16ac205ec6f241d69d5e6b8408f09f38&role.id=5146b19817b54113be3f81111ef9d787 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:32.281071 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4286ba0-93ec-452e-84b4-7c066c666705 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:32.281354 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4286ba0-93ec-452e-84b4-7c066c666705 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:32.281623 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4286ba0-93ec-452e-84b4-7c066c666705 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:32.286293 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 242/490] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:32.291074 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade6dc8-b3b7-4a53-b36c-1d5b108b3707 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:32.291354 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade6dc8-b3b7-4a53-b36c-1d5b108b3707 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:32.291622 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade6dc8-b3b7-4a53-b36c-1d5b108b3707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:32.315601 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d3e69-6f84-40d2-a8b9-ba46b7a4e4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:32.316640 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d3e69-6f84-40d2-a8b9-ba46b7a4e4f2 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:32.317117 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d3e69-6f84-40d2-a8b9-ba46b7a4e4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:32.317583 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d3e69-6f84-40d2-a8b9-ba46b7a4e4f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:32.325540 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-2ade6dc8-b3b7-4a53-b36c-1d5b108b3707 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:32.326940 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2ade6dc8-b3b7-4a53-b36c-1d5b108b3707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:32.330271 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d3e69-6f84-40d2-a8b9-ba46b7a4e4f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:24:32.336464 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-060d3e69-6f84-40d2-a8b9-ba46b7a4e4f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 09:24:32.336464 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:32.336464 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:32.336464 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:32.336464 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:32.336464 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:32.336464 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:32.336464 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:32.336464 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:32.336464 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:32.336464 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:32.336464 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:32.336464 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:24:32.336464 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:24:32.336464 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:24:32.336464 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:24:32.337375 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:24:32.337375 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:24:32.337375 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:24:32.337375 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:24:32.337375 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:32.337375 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:32.337375 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:32.337375 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:32.337375 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:32.337375 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:32.337375 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:32.337375 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:32.337375 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:32.337375 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:32.337375 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:32.337375 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:32.338167 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:32.338167 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:32.338167 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:32.338167 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:32.338167 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:24:32.338167 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:24:32.338167 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:24:32.338167 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:24:32.338167 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:24:32.338167 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:24:32.338167 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 09:24:32.338167 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:32.340922 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 249/491] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Oct 27 09:24:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 09:24:32.347237 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d3e69-6f84-40d2-a8b9-ba46b7a4e4f2 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:32.393980 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:32.459086 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d3e69-6f84-40d2-a8b9-ba46b7a4e4f2 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:32.461588 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 243/492] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 27 09:24:32.497584 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea22457a-ef10-4f4b-ac1b-85e7d071ec1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:32.498579 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea22457a-ef10-4f4b-ac1b-85e7d071ec1b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:32.499072 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea22457a-ef10-4f4b-ac1b-85e7d071ec1b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:32.499536 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea22457a-ef10-4f4b-ac1b-85e7d071ec1b None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:32.500402 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea22457a-ef10-4f4b-ac1b-85e7d071ec1b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:32.500927 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea22457a-ef10-4f4b-ac1b-85e7d071ec1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:32.502482 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea22457a-ef10-4f4b-ac1b-85e7d071ec1b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:32.512061 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 250/493] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Oct 27 09:24:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:32.518055 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade6dc8-b3b7-4a53-b36c-1d5b108b3707 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:32.647792 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:32.660040 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade6dc8-b3b7-4a53-b36c-1d5b108b3707 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:32.702901 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044b1852-cb7b-4d7c-8cc8-b6d80e215a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:32.703580 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b1852-cb7b-4d7c-8cc8-b6d80e215a6e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:32.703865 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b1852-cb7b-4d7c-8cc8-b6d80e215a6e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:32.704137 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b1852-cb7b-4d7c-8cc8-b6d80e215a6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:32.714999 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044b1852-cb7b-4d7c-8cc8-b6d80e215a6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:32.716456 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044b1852-cb7b-4d7c-8cc8-b6d80e215a6e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:32.732767 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-044b1852-cb7b-4d7c-8cc8-b6d80e215a6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:32.732767 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:32.732767 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:32.732767 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:32.732767 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:32.732767 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:32.732767 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:32.732767 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:32.732767 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:32.732767 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:32.732767 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:32.732767 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:32.732767 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:24:32.732767 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:24:32.732767 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:24:32.732767 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:24:32.734101 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:32.734101 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:32.734101 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:32.734101 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:32.734101 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:32.734101 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:32.734101 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:32.734101 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:32.734101 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:32.734101 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:32.734101 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:32.734101 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:32.734101 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:32.734101 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:32.734101 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:32.734101 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:32.734101 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:24:32.735186 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:24:32.735186 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:32.735186 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:32.735186 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 244/494] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:24:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:32.742059 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea22457a-ef10-4f4b-ac1b-85e7d071ec1b None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:32.799037 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044b1852-cb7b-4d7c-8cc8-b6d80e215a6e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:32.855050 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea22457a-ef10-4f4b-ac1b-85e7d071ec1b None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:32.857339 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:32.895261 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1b3156-2908-44d1-a449-c320014df310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:32.896139 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1b3156-2908-44d1-a449-c320014df310 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:32.896618 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1b3156-2908-44d1-a449-c320014df310 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:32.897027 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1b3156-2908-44d1-a449-c320014df310 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:32.898380 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1b3156-2908-44d1-a449-c320014df310 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:32.898838 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1b3156-2908-44d1-a449-c320014df310 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:32.900551 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1b3156-2908-44d1-a449-c320014df310 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:32.914457 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 251/495] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:32.920225 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1b3156-2908-44d1-a449-c320014df310 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:32.936963 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044b1852-cb7b-4d7c-8cc8-b6d80e215a6e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:32.975276 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a192c501-88cd-4072-963d-24298556ba73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:32.975945 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a192c501-88cd-4072-963d-24298556ba73 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:32.976231 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a192c501-88cd-4072-963d-24298556ba73 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:32.976508 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a192c501-88cd-4072-963d-24298556ba73 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:32.987614 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a192c501-88cd-4072-963d-24298556ba73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:24:32.993280 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-a192c501-88cd-4072-963d-24298556ba73 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 27 09:24:32.993280 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:32.993280 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:32.993280 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:32.993280 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:32.993280 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:32.993280 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:32.993280 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:32.993280 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:32.993280 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:32.993280 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:32.993280 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:32.993280 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:24:32.993280 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:24:32.993280 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:24:32.993280 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:24:32.994181 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:24:32.994181 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:24:32.994181 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:24:32.994181 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:24:32.994181 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:32.994181 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:32.994181 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:32.994181 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:32.994181 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:32.994181 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:32.994181 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:32.994181 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:32.994181 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:32.994181 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:32.994181 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:32.994181 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:32.994970 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:32.994970 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:32.994970 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:32.994970 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:32.994970 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:24:32.994970 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:24:32.994970 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:24:32.994970 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:24:32.994970 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:24:32.994970 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:24:32.994970 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 27 09:24:32.994970 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:32.999475 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 245/496] 10.222.0.70 () {56 vars in 1102 bytes} [Thu Oct 27 09:24:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 09:24:33.000878 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a192c501-88cd-4072-963d-24298556ba73 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:33.020891 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:33.151699 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1b3156-2908-44d1-a449-c320014df310 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:33.156209 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a192c501-88cd-4072-963d-24298556ba73 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:33.205708 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8c9b48-9d92-46ee-a5cc-7d1c7f2e0950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:33.206524 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c9b48-9d92-46ee-a5cc-7d1c7f2e0950 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:33.206810 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c9b48-9d92-46ee-a5cc-7d1c7f2e0950 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:33.207081 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c9b48-9d92-46ee-a5cc-7d1c7f2e0950 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:33.207803 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8c9b48-9d92-46ee-a5cc-7d1c7f2e0950 None admin] RBAC: Adding query filter params (name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:33.208105 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8c9b48-9d92-46ee-a5cc-7d1c7f2e0950 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:33.209702 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8c9b48-9d92-46ee-a5cc-7d1c7f2e0950 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:33.226599 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 246/497] 10.222.0.70 () {56 vars in 1103 bytes} [Thu Oct 27 09:24:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:33.228731 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8c9b48-9d92-46ee-a5cc-7d1c7f2e0950 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:33.230132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d65d2c-9f4e-4951-a3d5-b4df6f096727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:33.231028 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d65d2c-9f4e-4951-a3d5-b4df6f096727 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:33.231504 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d65d2c-9f4e-4951-a3d5-b4df6f096727 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:33.231970 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d65d2c-9f4e-4951-a3d5-b4df6f096727 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:33.249975 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d65d2c-9f4e-4951-a3d5-b4df6f096727 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:33.251848 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d65d2c-9f4e-4951-a3d5-b4df6f096727 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:33.278826 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-06d65d2c-9f4e-4951-a3d5-b4df6f096727 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 27 09:24:33.278826 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:33.278826 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:33.278826 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:33.278826 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:33.278826 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:33.278826 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:33.278826 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:33.278826 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:33.278826 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:33.278826 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:33.278826 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:33.278826 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 09:24:33.278826 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 09:24:33.278826 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 09:24:33.278826 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 09:24:33.279787 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:33.279787 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:33.279787 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 09:24:33.279787 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:24:33.279787 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 09:24:33.279787 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:24:33.279787 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:33.279787 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:33.279787 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:33.279787 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:33.279787 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:33.279787 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:33.279787 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:33.279787 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:33.279787 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:33.279787 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:33.279787 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:33.279787 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:33.280814 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:33.280814 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:33.280814 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 09:24:33.280814 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 09:24:33.280814 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 09:24:33.280814 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 09:24:33.280814 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 09:24:33.280814 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 09:24:33.280814 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 27 09:24:33.280814 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:33.282131 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 252/498] 10.222.0.70 () {56 vars in 1093 bytes} [Thu Oct 27 09:24:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:33.288240 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d65d2c-9f4e-4951-a3d5-b4df6f096727 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:33.336689 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:33.339807 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:33.384871 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8c9b48-9d92-46ee-a5cc-7d1c7f2e0950 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:33.424235 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732f2a12-e6fc-4f12-ba93-989191fd5318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:33.424853 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f2a12-e6fc-4f12-ba93-989191fd5318 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:33.425149 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f2a12-e6fc-4f12-ba93-989191fd5318 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:33.425419 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f2a12-e6fc-4f12-ba93-989191fd5318 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:33.437852 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732f2a12-e6fc-4f12-ba93-989191fd5318 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:33.439320 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732f2a12-e6fc-4f12-ba93-989191fd5318 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:33.455675 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-732f2a12-e6fc-4f12-ba93-989191fd5318 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:33.455675 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:33.455675 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:33.455675 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:33.455675 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:33.455675 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:33.455675 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:33.455675 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:33.455675 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:33.455675 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:33.455675 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:33.455675 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:33.455675 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:24:33.455675 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:24:33.455675 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:24:33.455675 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:24:33.456695 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:33.456695 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:33.456695 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:33.456695 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:33.456695 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:33.456695 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:33.456695 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:33.456695 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:33.456695 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:33.456695 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:33.456695 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:33.456695 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:33.456695 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:33.456695 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:33.456695 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:33.456695 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:33.456695 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:24:33.458065 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:24:33.458065 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:33.458065 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:33.461695 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 247/499] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:24:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 09:24:33.463910 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732f2a12-e6fc-4f12-ba93-989191fd5318 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:33.509847 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d65d2c-9f4e-4951-a3d5-b4df6f096727 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:33.546456 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa3e86-751b-40e6-a71b-b476d831ca8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:33.547072 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa3e86-751b-40e6-a71b-b476d831ca8a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:33.547348 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa3e86-751b-40e6-a71b-b476d831ca8a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:33.547619 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa3e86-751b-40e6-a71b-b476d831ca8a None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:33.548333 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fa3e86-751b-40e6-a71b-b476d831ca8a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:33.548680 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fa3e86-751b-40e6-a71b-b476d831ca8a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:33.550099 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fa3e86-751b-40e6-a71b-b476d831ca8a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:33.572693 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 253/500] 10.222.0.70 () {56 vars in 1133 bytes} [Thu Oct 27 09:24:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:33.574808 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b4598-173d-4e43-bdda-04d8f7f1a390 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:33.575097 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b4598-173d-4e43-bdda-04d8f7f1a390 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:33.575372 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b4598-173d-4e43-bdda-04d8f7f1a390 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:33.577352 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 254/501] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:33.581348 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b4598-173d-4e43-bdda-04d8f7f1a390 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:33.600406 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732f2a12-e6fc-4f12-ba93-989191fd5318 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:33.620351 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:33.639531 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c9a82-2dfb-47dd-920b-62266c528503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:33.640266 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c9a82-2dfb-47dd-920b-62266c528503 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:33.640690 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c9a82-2dfb-47dd-920b-62266c528503 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:33.641131 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c9a82-2dfb-47dd-920b-62266c528503 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:33.641873 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424c9a82-2dfb-47dd-920b-62266c528503 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:33.642368 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424c9a82-2dfb-47dd-920b-62266c528503 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:33.643852 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424c9a82-2dfb-47dd-920b-62266c528503 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:33.657425 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 248/502] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:33.658852 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be9140-eff6-4a96-b8e3-5e1853f5d71b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:33.659232 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be9140-eff6-4a96-b8e3-5e1853f5d71b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:33.659654 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be9140-eff6-4a96-b8e3-5e1853f5d71b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:33.683391 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-14be9140-eff6-4a96-b8e3-5e1853f5d71b None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:33.685316 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-14be9140-eff6-4a96-b8e3-5e1853f5d71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:33.697296 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:33.724002 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b4598-173d-4e43-bdda-04d8f7f1a390 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:33.761087 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 249/503] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:24:33.762068 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14be9140-eff6-4a96-b8e3-5e1853f5d71b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:33.779536 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c045df9f-f82c-485e-ae3f-22f6e8a0f0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:33.780172 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045df9f-f82c-485e-ae3f-22f6e8a0f0ef None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:33.780495 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045df9f-f82c-485e-ae3f-22f6e8a0f0ef None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:33.780798 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045df9f-f82c-485e-ae3f-22f6e8a0f0ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:33.801206 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c045df9f-f82c-485e-ae3f-22f6e8a0f0ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:33.804346 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c045df9f-f82c-485e-ae3f-22f6e8a0f0ef None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:33.819770 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-c045df9f-f82c-485e-ae3f-22f6e8a0f0ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:33.819770 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:33.819770 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:33.819770 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:33.819770 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:33.819770 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:33.819770 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:33.819770 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:33.819770 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:33.819770 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:33.819770 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:33.819770 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:33.819770 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:24:33.819770 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:24:33.819770 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:24:33.819770 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:24:33.821667 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:33.821667 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:33.821667 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:33.821667 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:33.821667 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:33.821667 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:33.821667 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:33.821667 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:33.821667 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:33.821667 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:33.821667 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:33.821667 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:33.821667 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:33.821667 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:33.821667 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:33.821667 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:33.821667 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:24:33.823187 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:24:33.823187 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:33.823187 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:33.823187 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 255/504] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:24:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:33.828735 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c045df9f-f82c-485e-ae3f-22f6e8a0f0ef None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:33.952141 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14be9140-eff6-4a96-b8e3-5e1853f5d71b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:33.970386 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:34.009741 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c045df9f-f82c-485e-ae3f-22f6e8a0f0ef None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:34.015279 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e12c2f-c768-44dd-8e3e-f11c1512ab56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:34.015913 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e12c2f-c768-44dd-8e3e-f11c1512ab56 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:34.016188 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e12c2f-c768-44dd-8e3e-f11c1512ab56 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:34.016452 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e12c2f-c768-44dd-8e3e-f11c1512ab56 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:34.030208 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e12c2f-c768-44dd-8e3e-f11c1512ab56 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:34.031669 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e12c2f-c768-44dd-8e3e-f11c1512ab56 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:34.058358 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-80e12c2f-c768-44dd-8e3e-f11c1512ab56 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 27 09:24:34.058358 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:34.058358 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:34.058358 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:34.058358 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:34.058358 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:34.058358 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:34.058358 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:34.058358 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:34.058358 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:34.058358 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:34.058358 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:34.058358 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 09:24:34.058358 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 09:24:34.058358 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 09:24:34.058358 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 09:24:34.060682 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:34.060682 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:34.060682 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 09:24:34.060682 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:24:34.060682 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 09:24:34.060682 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:24:34.060682 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:34.060682 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:34.060682 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:34.060682 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:34.060682 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:34.060682 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:34.060682 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:34.060682 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:34.060682 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:34.060682 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:34.060682 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:34.060682 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:34.062990 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:34.062990 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:34.062990 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 09:24:34.062990 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 09:24:34.062990 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 09:24:34.062990 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 09:24:34.062990 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 09:24:34.062990 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 09:24:34.062990 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 27 09:24:34.062990 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:34.062990 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 250/505] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Oct 27 09:24:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:34.068963 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e12c2f-c768-44dd-8e3e-f11c1512ab56 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:34.072048 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:34.077936 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf45a473-c20c-40f8-a98b-ed26e1fc4295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:34.077936 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45a473-c20c-40f8-a98b-ed26e1fc4295 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:34.077936 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45a473-c20c-40f8-a98b-ed26e1fc4295 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:34.085470 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45a473-c20c-40f8-a98b-ed26e1fc4295 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:34.086155 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf45a473-c20c-40f8-a98b-ed26e1fc4295 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:34.086471 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf45a473-c20c-40f8-a98b-ed26e1fc4295 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:34.087869 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf45a473-c20c-40f8-a98b-ed26e1fc4295 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:34.123119 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 256/506] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:24:34.124218 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf45a473-c20c-40f8-a98b-ed26e1fc4295 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:34.184290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e12c2f-c768-44dd-8e3e-f11c1512ab56 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:34.223065 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d4e7e6-1dbf-4627-9c0c-54012d2bfe49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:34.224001 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4e7e6-1dbf-4627-9c0c-54012d2bfe49 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:34.224480 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4e7e6-1dbf-4627-9c0c-54012d2bfe49 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:34.224979 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4e7e6-1dbf-4627-9c0c-54012d2bfe49 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:34.225913 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d4e7e6-1dbf-4627-9c0c-54012d2bfe49 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:34.226417 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d4e7e6-1dbf-4627-9c0c-54012d2bfe49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:34.228122 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d4e7e6-1dbf-4627-9c0c-54012d2bfe49 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:34.249509 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf45a473-c20c-40f8-a98b-ed26e1fc4295 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:34.249838 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 251/507] 10.222.0.70 () {56 vars in 1135 bytes} [Thu Oct 27 09:24:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:34.250851 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d4e7e6-1dbf-4627-9c0c-54012d2bfe49 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:34.280271 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:34.301054 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea959bd-aedd-466f-a105-5ba11b48c94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:34.301803 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea959bd-aedd-466f-a105-5ba11b48c94a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:34.302107 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea959bd-aedd-466f-a105-5ba11b48c94a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:34.302959 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea959bd-aedd-466f-a105-5ba11b48c94a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:34.311648 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea959bd-aedd-466f-a105-5ba11b48c94a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:34.313473 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea959bd-aedd-466f-a105-5ba11b48c94a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:34.328487 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-6ea959bd-aedd-466f-a105-5ba11b48c94a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:24:34.328487 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:34.328487 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:34.328487 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:34.328487 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:34.328487 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:34.328487 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:34.328487 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:34.328487 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:34.328487 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:34.328487 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:34.328487 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:34.328487 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 09:24:34.328487 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 09:24:34.328487 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 09:24:34.329401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 09:24:34.329401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:34.329401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:34.329401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:34.329401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:34.329401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:34.329401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:34.329401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:34.329401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:34.329401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:34.329401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:34.329401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:34.329401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:34.329401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:34.329401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:34.329401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:34.329401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:34.330272 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 09:24:34.330272 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 09:24:34.330272 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 09:24:34.330272 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 09:24:34.330272 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 09:24:34.330272 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 09:24:34.330272 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:24:34.330272 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:34.332596 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 257/508] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Oct 27 09:24:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 09:24:34.333619 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea959bd-aedd-466f-a105-5ba11b48c94a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:34.375341 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d4e7e6-1dbf-4627-9c0c-54012d2bfe49 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:34.407327 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:34.414728 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f288f8c2-7ba8-45f0-8810-1fcc089a7c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:34.415622 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f288f8c2-7ba8-45f0-8810-1fcc089a7c63 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:34.416174 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f288f8c2-7ba8-45f0-8810-1fcc089a7c63 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:34.416774 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f288f8c2-7ba8-45f0-8810-1fcc089a7c63 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:34.417567 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f288f8c2-7ba8-45f0-8810-1fcc089a7c63 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:34.419314 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f288f8c2-7ba8-45f0-8810-1fcc089a7c63 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:34.430783 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-f288f8c2-7ba8-45f0-8810-1fcc089a7c63 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 27 09:24:34.430783 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:34.430783 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 09:24:34.430783 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 09:24:34.430783 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 09:24:34.430783 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 09:24:34.430783 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 09:24:34.430783 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:24:34.430783 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 27 09:24:34.430783 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:34.430783 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 09:24:34.430783 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:34.430783 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:34.430783 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:34.430783 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:34.430783 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:34.430783 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:34.431809 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:34.431809 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:34.431809 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:34.431809 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:34.431809 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:34.431809 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:34.431809 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 09:24:34.431809 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 09:24:34.431809 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 09:24:34.431809 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 09:24:34.431809 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:34.431809 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:34.431809 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:34.431809 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:34.431809 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:34.431809 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:34.432667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:34.432667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:34.432667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:34.432667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:34.432667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:34.432667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:34.432667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:34.432667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:34.432667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:34.432667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:34.432667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 09:24:34.432667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:24:34.432667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 27 09:24:34.432667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:34.435589 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 252/509] 10.222.0.70 () {56 vars in 1129 bytes} [Thu Oct 27 09:24:34 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:34.436610 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f288f8c2-7ba8-45f0-8810-1fcc089a7c63 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:34.467341 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea959bd-aedd-466f-a105-5ba11b48c94a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:34.502166 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f571b1-14b8-42b8-abaf-8f48c7a95668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:34.502690 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f571b1-14b8-42b8-abaf-8f48c7a95668 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:34.502961 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f571b1-14b8-42b8-abaf-8f48c7a95668 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:34.503227 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f571b1-14b8-42b8-abaf-8f48c7a95668 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:34.546383 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:34.556636 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f288f8c2-7ba8-45f0-8810-1fcc089a7c63 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:34.573870 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f571b1-14b8-42b8-abaf-8f48c7a95668 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:34.575312 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f571b1-14b8-42b8-abaf-8f48c7a95668 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:34.592837 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31051edd-21d6-4e98-905b-5e302e42674e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:34.593577 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31051edd-21d6-4e98-905b-5e302e42674e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:34.593868 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31051edd-21d6-4e98-905b-5e302e42674e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:34.594123 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31051edd-21d6-4e98-905b-5e302e42674e None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:34.595084 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31051edd-21d6-4e98-905b-5e302e42674e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:34.595363 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31051edd-21d6-4e98-905b-5e302e42674e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:34.596944 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31051edd-21d6-4e98-905b-5e302e42674e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:34.614686 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 253/510] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Oct 27 09:24:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:34.614686 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31051edd-21d6-4e98-905b-5e302e42674e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:34.629239 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:34.687595 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 258/511] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Oct 27 09:24:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 27 09:24:34.687595 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785efb5-71a6-4d7b-aab5-4844e93dd370 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:34.687595 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785efb5-71a6-4d7b-aab5-4844e93dd370 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:34.687595 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785efb5-71a6-4d7b-aab5-4844e93dd370 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:34.688648 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 259/512] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:34.689659 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6785efb5-71a6-4d7b-aab5-4844e93dd370 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:34.725918 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31051edd-21d6-4e98-905b-5e302e42674e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:34.763247 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc3b938-402f-4014-b4c2-aa22bee97d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:34.763827 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc3b938-402f-4014-b4c2-aa22bee97d53 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:34.764109 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc3b938-402f-4014-b4c2-aa22bee97d53 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:34.764375 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc3b938-402f-4014-b4c2-aa22bee97d53 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:34.765126 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc3b938-402f-4014-b4c2-aa22bee97d53 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:34.765424 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc3b938-402f-4014-b4c2-aa22bee97d53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:34.766905 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc3b938-402f-4014-b4c2-aa22bee97d53 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:34.776310 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 254/513] 10.222.0.70 () {56 vars in 1124 bytes} [Thu Oct 27 09:24:34 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:34.776310 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc3b938-402f-4014-b4c2-aa22bee97d53 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:34.792392 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:34.806265 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6785efb5-71a6-4d7b-aab5-4844e93dd370 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:34.841579 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8384062-b072-44c0-8ca3-51d6bd9981eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:34.842249 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8384062-b072-44c0-8ca3-51d6bd9981eb None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:34.842524 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8384062-b072-44c0-8ca3-51d6bd9981eb None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:34.842793 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8384062-b072-44c0-8ca3-51d6bd9981eb None admin] PATH_INFO: `/v3/projects/16ac205ec6f241d69d5e6b8408f09f38/users/dbe4a6e4b6184826b30ac4fc13ad9b1f/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:34.845101 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8384062-b072-44c0-8ca3-51d6bd9981eb None admin] RBAC: Authorizing `identity:create_grant(project_id=16ac205ec6f241d69d5e6b8408f09f38, user_id=dbe4a6e4b6184826b30ac4fc13ad9b1f, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:34.846566 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8384062-b072-44c0-8ca3-51d6bd9981eb None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:34.864233 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 260/514] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Oct 27 09:24:34 2022] PUT /identity/v3/projects/16ac205ec6f241d69d5e6b8408f09f38/users/dbe4a6e4b6184826b30ac4fc13ad9b1f/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:24:34.867195 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f21756c-df9b-46ba-81f2-dc261dad71f5 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:34.867865 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f21756c-df9b-46ba-81f2-dc261dad71f5 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:34.868397 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f21756c-df9b-46ba-81f2-dc261dad71f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:34.890015 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-9f21756c-df9b-46ba-81f2-dc261dad71f5 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:34.892265 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9f21756c-df9b-46ba-81f2-dc261dad71f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:34.893625 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc3b938-402f-4014-b4c2-aa22bee97d53 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:34.905635 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:34.938187 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0e63b4-2b71-4976-87bf-a8a700253075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:34.939127 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e63b4-2b71-4976-87bf-a8a700253075 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:34.939479 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e63b4-2b71-4976-87bf-a8a700253075 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:34.939809 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e63b4-2b71-4976-87bf-a8a700253075 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:34.953250 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0e63b4-2b71-4976-87bf-a8a700253075 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:34.954858 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0e63b4-2b71-4976-87bf-a8a700253075 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:34.972711 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-0d0e63b4-2b71-4976-87bf-a8a700253075 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:34.972711 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:34.972711 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:34.972711 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:34.972711 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:34.972711 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:34.972711 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:34.972711 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:34.972711 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:34.972711 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:34.972711 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:34.972711 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:34.972711 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:24:34.972711 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:24:34.972711 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:24:34.972711 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:24:34.973724 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:34.973724 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:34.973724 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:34.973724 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:34.973724 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:34.973724 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:34.973724 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:34.973724 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:34.973724 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:34.973724 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:34.973724 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:34.973724 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:34.973724 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:34.973724 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:34.973724 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:34.973724 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:34.973724 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:24:34.974594 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:24:34.974594 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:34.974594 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:34.974594 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 255/515] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:24:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:34.977345 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0e63b4-2b71-4976-87bf-a8a700253075 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:35.000683 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 261/516] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:24:35.002124 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c286e-8463-4d9c-8007-db319174736f None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:35.002412 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c286e-8463-4d9c-8007-db319174736f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:35.002679 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c286e-8463-4d9c-8007-db319174736f None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:35.005363 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 262/517] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:35.005363 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7c286e-8463-4d9c-8007-db319174736f None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:35.065183 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:35.108038 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0e63b4-2b71-4976-87bf-a8a700253075 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:35.137512 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7c286e-8463-4d9c-8007-db319174736f None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:35.144162 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398a63db-0734-4646-bc0c-0580d3edfbda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:35.145288 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398a63db-0734-4646-bc0c-0580d3edfbda None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:35.145720 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398a63db-0734-4646-bc0c-0580d3edfbda None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:35.146141 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398a63db-0734-4646-bc0c-0580d3edfbda None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:35.146964 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398a63db-0734-4646-bc0c-0580d3edfbda None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:35.147556 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398a63db-0734-4646-bc0c-0580d3edfbda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:35.149103 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398a63db-0734-4646-bc0c-0580d3edfbda None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:35.160848 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 256/518] 10.222.0.70 () {56 vars in 1124 bytes} [Thu Oct 27 09:24:34 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:35.162516 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d09375-ee7c-483c-a17a-1c7e80441963 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:35.163529 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d09375-ee7c-483c-a17a-1c7e80441963 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:35.163950 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d09375-ee7c-483c-a17a-1c7e80441963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:35.176215 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ab798-08bc-458e-8f3f-a41e73233589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:35.177025 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ab798-08bc-458e-8f3f-a41e73233589 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:35.177400 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ab798-08bc-458e-8f3f-a41e73233589 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:35.177773 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ab798-08bc-458e-8f3f-a41e73233589 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:35.179758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ab798-08bc-458e-8f3f-a41e73233589 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:35.179758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ab798-08bc-458e-8f3f-a41e73233589 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:35.181106 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ab798-08bc-458e-8f3f-a41e73233589 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:35.193708 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 263/519] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:24:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:35.201685 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ab798-08bc-458e-8f3f-a41e73233589 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:35.202984 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-e5d09375-ee7c-483c-a17a-1c7e80441963 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:35.207191 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e5d09375-ee7c-483c-a17a-1c7e80441963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:35.209475 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:35.321853 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 257/520] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:24:35.322753 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d09375-ee7c-483c-a17a-1c7e80441963 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:35.337417 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ab798-08bc-458e-8f3f-a41e73233589 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:35.398543 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041d00c-315e-47fd-a6c2-df1cadf28829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:35.399248 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041d00c-315e-47fd-a6c2-df1cadf28829 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:35.399534 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041d00c-315e-47fd-a6c2-df1cadf28829 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:35.399806 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041d00c-315e-47fd-a6c2-df1cadf28829 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:35.410486 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f041d00c-315e-47fd-a6c2-df1cadf28829 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:35.411447 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:35.412674 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f041d00c-315e-47fd-a6c2-df1cadf28829 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:35.427159 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-f041d00c-315e-47fd-a6c2-df1cadf28829 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:24:35.427159 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:35.427159 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:35.427159 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:35.427159 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:35.427159 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:35.427159 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:35.427159 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:35.427159 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:35.427159 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:35.427159 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:35.427159 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:35.427159 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 09:24:35.427159 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 09:24:35.427159 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 09:24:35.428072 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 09:24:35.428072 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:35.428072 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:35.428072 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:35.428072 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:35.428072 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:35.428072 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:35.428072 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:35.428072 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:35.428072 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:35.428072 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:35.428072 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:35.428072 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:35.428072 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:35.428072 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:35.428072 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:35.428072 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:35.429181 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 09:24:35.429181 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 09:24:35.429181 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 09:24:35.429181 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 09:24:35.429181 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 09:24:35.429181 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 09:24:35.429181 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:24:35.429181 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:35.429181 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 264/521] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Oct 27 09:24:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:35.438601 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041d00c-315e-47fd-a6c2-df1cadf28829 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:35.474528 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d09375-ee7c-483c-a17a-1c7e80441963 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:35.526663 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a542d-ce0f-4f84-9261-a9384de0e9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:35.527294 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a542d-ce0f-4f84-9261-a9384de0e9af None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:35.527572 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a542d-ce0f-4f84-9261-a9384de0e9af None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:35.527842 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a542d-ce0f-4f84-9261-a9384de0e9af None admin] PATH_INFO: `/v3/services/network` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:35.528425 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075a542d-ce0f-4f84-9261-a9384de0e9af None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:35.529824 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075a542d-ce0f-4f84-9261-a9384de0e9af None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:35.548635 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-075a542d-ce0f-4f84-9261-a9384de0e9af None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 27 09:24:35.548635 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:35.548635 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 09:24:35.548635 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 09:24:35.548635 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 09:24:35.548635 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 09:24:35.548635 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 09:24:35.548635 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:24:35.548635 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 27 09:24:35.548635 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:35.548635 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 09:24:35.548635 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:35.548635 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:35.548635 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:35.548635 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:35.548635 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:35.548635 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:35.549714 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:35.549714 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:35.549714 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:35.549714 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:35.549714 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:35.549714 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:35.549714 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 09:24:35.549714 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 09:24:35.549714 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 09:24:35.549714 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 09:24:35.549714 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:35.549714 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:35.549714 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:35.549714 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:35.549714 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:35.549714 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:35.550551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:35.550551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:35.550551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:35.550551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:35.550551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:35.550551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:35.550551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:35.550551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:35.550551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:35.550551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:35.550551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 09:24:35.550551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:24:35.550551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 27 09:24:35.550551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:35.552055 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 258/522] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Oct 27 09:24:35 2022] GET /identity/v3/services/network => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:35.557821 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a542d-ce0f-4f84-9261-a9384de0e9af None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:35.558324 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:35.610461 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041d00c-315e-47fd-a6c2-df1cadf28829 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:35.649913 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018a4b33-1512-44a9-9897-bcb041ddada8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:35.650850 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018a4b33-1512-44a9-9897-bcb041ddada8 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:35.651134 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018a4b33-1512-44a9-9897-bcb041ddada8 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:35.651404 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018a4b33-1512-44a9-9897-bcb041ddada8 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:35.652146 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018a4b33-1512-44a9-9897-bcb041ddada8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:35.652490 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018a4b33-1512-44a9-9897-bcb041ddada8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:35.654154 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018a4b33-1512-44a9-9897-bcb041ddada8 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:35.672378 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 265/523] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Oct 27 09:24:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:35.673276 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018a4b33-1512-44a9-9897-bcb041ddada8 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:35.695376 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a542d-ce0f-4f84-9261-a9384de0e9af None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:35.727001 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:35.740961 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41f08d4-869c-46db-9d86-5294d506decc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:35.742791 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f08d4-869c-46db-9d86-5294d506decc None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:35.743415 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f08d4-869c-46db-9d86-5294d506decc None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:35.744417 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f08d4-869c-46db-9d86-5294d506decc None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:35.745216 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41f08d4-869c-46db-9d86-5294d506decc None admin] RBAC: Adding query filter params (name=network) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:35.745860 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41f08d4-869c-46db-9d86-5294d506decc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:35.748005 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41f08d4-869c-46db-9d86-5294d506decc None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:35.758020 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 259/524] 10.222.0.70 () {56 vars in 1112 bytes} [Thu Oct 27 09:24:35 2022] GET /identity/v3/services?name=network => generated 124 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:35.759025 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41f08d4-869c-46db-9d86-5294d506decc None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:35.803864 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018a4b33-1512-44a9-9897-bcb041ddada8 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:35.831879 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:35.846738 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa4d31-4a07-448b-9859-84257aa4fbc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:35.847442 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa4d31-4a07-448b-9859-84257aa4fbc6 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:35.847918 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa4d31-4a07-448b-9859-84257aa4fbc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:35.848194 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa4d31-4a07-448b-9859-84257aa4fbc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:35.860424 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fa4d31-4a07-448b-9859-84257aa4fbc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:35.861746 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fa4d31-4a07-448b-9859-84257aa4fbc6 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:35.878777 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-12fa4d31-4a07-448b-9859-84257aa4fbc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:35.878777 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:35.878777 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:35.878777 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:35.878777 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:35.878777 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:35.878777 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:35.878777 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:35.878777 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:35.878777 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:35.878777 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:35.878777 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:35.878777 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:24:35.878777 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:24:35.878777 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:24:35.878777 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:24:35.879760 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:35.879760 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:35.879760 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:35.879760 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:35.879760 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:35.879760 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:35.879760 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:35.879760 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:35.879760 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:35.879760 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:35.879760 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:35.879760 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:35.879760 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:35.879760 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:35.879760 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:35.879760 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:35.879760 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:24:35.880674 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:24:35.880674 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:35.880674 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:35.884595 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 266/525] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:24:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:35.885611 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa4d31-4a07-448b-9859-84257aa4fbc6 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:35.896656 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41f08d4-869c-46db-9d86-5294d506decc None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:35.948754 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc794f-c29e-4c5c-b4f0-edf814a864f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:35.949731 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc794f-c29e-4c5c-b4f0-edf814a864f6 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:35.950232 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc794f-c29e-4c5c-b4f0-edf814a864f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:35.950703 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc794f-c29e-4c5c-b4f0-edf814a864f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:35.952229 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc794f-c29e-4c5c-b4f0-edf814a864f6 None admin] RBAC: Adding query filter params (role.id=5146b19817b54113be3f81111ef9d787, scope.project.id=16ac205ec6f241d69d5e6b8408f09f38, user.id=eecf79ea4700496c89b1a67c614b34d0) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:35.953064 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc794f-c29e-4c5c-b4f0-edf814a864f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:35.955388 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc794f-c29e-4c5c-b4f0-edf814a864f6 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:35.964775 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 260/526] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Oct 27 09:24:35 2022] GET /identity/v3/role_assignments?user.id=eecf79ea4700496c89b1a67c614b34d0&scope.project.id=16ac205ec6f241d69d5e6b8408f09f38&role.id=5146b19817b54113be3f81111ef9d787 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:35.966083 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc794f-c29e-4c5c-b4f0-edf814a864f6 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:36.036930 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa4d31-4a07-448b-9859-84257aa4fbc6 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:36.084217 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:36.135932 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58757c52-bc2f-4743-9072-828587cef3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:36.136469 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58757c52-bc2f-4743-9072-828587cef3ec None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:36.136837 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58757c52-bc2f-4743-9072-828587cef3ec None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:36.137113 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58757c52-bc2f-4743-9072-828587cef3ec None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:36.137744 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58757c52-bc2f-4743-9072-828587cef3ec None admin] RBAC: Adding query filter params (type=network) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:36.138263 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58757c52-bc2f-4743-9072-828587cef3ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:36.139406 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58757c52-bc2f-4743-9072-828587cef3ec None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:36.149838 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 267/527] 10.222.0.70 () {56 vars in 1112 bytes} [Thu Oct 27 09:24:35 2022] GET /identity/v3/services?type=network => generated 124 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:36.170866 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc794f-c29e-4c5c-b4f0-edf814a864f6 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:36.209419 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493dbbf4-ddaa-4d7d-becb-a470f58a848d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:36.210086 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493dbbf4-ddaa-4d7d-becb-a470f58a848d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:36.210377 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493dbbf4-ddaa-4d7d-becb-a470f58a848d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:36.210636 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493dbbf4-ddaa-4d7d-becb-a470f58a848d None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:36.211295 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493dbbf4-ddaa-4d7d-becb-a470f58a848d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:36.211595 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493dbbf4-ddaa-4d7d-becb-a470f58a848d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:36.213050 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493dbbf4-ddaa-4d7d-becb-a470f58a848d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:36.227363 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 261/528] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:24:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:36.232950 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58757c52-bc2f-4743-9072-828587cef3ec None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:36.294698 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:36.349954 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58757c52-bc2f-4743-9072-828587cef3ec None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:36.387445 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fa07f2-52bb-4d81-84d9-8e8985029d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:36.387975 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa07f2-52bb-4d81-84d9-8e8985029d4f None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:36.388282 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa07f2-52bb-4d81-84d9-8e8985029d4f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:36.388580 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa07f2-52bb-4d81-84d9-8e8985029d4f None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:36.389293 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fa07f2-52bb-4d81-84d9-8e8985029d4f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:36.390714 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fa07f2-52bb-4d81-84d9-8e8985029d4f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:36.417124 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 268/529] 10.222.0.70 () {60 vars in 1132 bytes} [Thu Oct 27 09:24:36 2022] POST /identity/v3/users => generated 254 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:24:37.401776 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb2b3b-4f2f-4fa9-9deb-94f15182a829 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:37.402297 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb2b3b-4f2f-4fa9-9deb-94f15182a829 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:37.402297 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb2b3b-4f2f-4fa9-9deb-94f15182a829 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:37.403926 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 262/530] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:37.409252 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35768fc3-d632-4467-8685-1ef2104de5d2 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:37.409648 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35768fc3-d632-4467-8685-1ef2104de5d2 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:37.409804 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35768fc3-d632-4467-8685-1ef2104de5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:37.433470 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-35768fc3-d632-4467-8685-1ef2104de5d2 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:37.434680 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-35768fc3-d632-4467-8685-1ef2104de5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:37.516858 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 269/531] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:24:37.788478 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af91c5-c494-4c7c-aa0d-d324b85bd648 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:37.788884 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af91c5-c494-4c7c-aa0d-d324b85bd648 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:37.789066 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af91c5-c494-4c7c-aa0d-d324b85bd648 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:37.790683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 263/532] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:37.796129 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9508d2-2c56-4088-9941-3ba1826e34c2 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:37.796417 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9508d2-2c56-4088-9941-3ba1826e34c2 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:37.796722 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9508d2-2c56-4088-9941-3ba1826e34c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:37.821434 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-ed9508d2-2c56-4088-9941-3ba1826e34c2 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:37.822626 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ed9508d2-2c56-4088-9941-3ba1826e34c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:37.845257 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:37.908540 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 270/533] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 09:24:37.949325 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75af91c5-c494-4c7c-aa0d-d324b85bd648 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:38.076122 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75af91c5-c494-4c7c-aa0d-d324b85bd648 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:38.091614 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:38.122493 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa04f57-945e-4802-9110-3a7834a3d725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:38.123192 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa04f57-945e-4802-9110-3a7834a3d725 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:38.123472 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa04f57-945e-4802-9110-3a7834a3d725 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:38.123738 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa04f57-945e-4802-9110-3a7834a3d725 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:38.137069 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa04f57-945e-4802-9110-3a7834a3d725 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:24:38.145762 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-eaa04f57-945e-4802-9110-3a7834a3d725 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 09:24:38.145762 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:38.145762 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:38.145762 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:38.145762 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:38.145762 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:38.145762 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:38.145762 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:38.145762 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:38.145762 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:38.145762 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:38.145762 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:38.145762 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:24:38.145762 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:24:38.145762 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:24:38.145762 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:24:38.146693 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:24:38.146693 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:24:38.146693 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:24:38.146693 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:24:38.146693 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:38.146693 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:38.146693 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:38.146693 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:38.146693 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:38.146693 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:38.146693 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:38.146693 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:38.146693 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:38.146693 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:38.146693 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:38.146693 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:38.148139 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:38.148139 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:38.148139 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:38.148139 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:38.148139 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:24:38.148139 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:24:38.148139 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:24:38.148139 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:24:38.148139 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:24:38.148139 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:24:38.148139 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 09:24:38.148139 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:38.148139 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 264/534] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Oct 27 09:24:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:38.153911 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9508d2-2c56-4088-9941-3ba1826e34c2 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:38.266985 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa04f57-945e-4802-9110-3a7834a3d725 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:38.330650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9508d2-2c56-4088-9941-3ba1826e34c2 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:38.393615 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b99aa-4d24-4eea-b8b0-646344ece107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:38.394231 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b99aa-4d24-4eea-b8b0-646344ece107 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:38.394508 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b99aa-4d24-4eea-b8b0-646344ece107 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:38.394784 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b99aa-4d24-4eea-b8b0-646344ece107 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:38.395445 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09b99aa-4d24-4eea-b8b0-646344ece107 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:38.395743 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09b99aa-4d24-4eea-b8b0-646344ece107 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:38.398403 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09b99aa-4d24-4eea-b8b0-646344ece107 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:38.414023 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 271/535] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Oct 27 09:24:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:38.420512 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b99aa-4d24-4eea-b8b0-646344ece107 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:38.441878 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa04f57-945e-4802-9110-3a7834a3d725 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:38.447758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:38.509140 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc98c40-624b-43dd-aee7-07356f8bfa8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:38.509746 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc98c40-624b-43dd-aee7-07356f8bfa8f None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:38.510022 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc98c40-624b-43dd-aee7-07356f8bfa8f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:38.510288 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc98c40-624b-43dd-aee7-07356f8bfa8f None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:38.510949 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc98c40-624b-43dd-aee7-07356f8bfa8f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:38.512311 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc98c40-624b-43dd-aee7-07356f8bfa8f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:38.540189 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 265/536] 10.222.0.70 () {60 vars in 1141 bytes} [Thu Oct 27 09:24:38 2022] POST /identity/v3/services => generated 251 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:24:38.588417 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b99aa-4d24-4eea-b8b0-646344ece107 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:38.646120 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b06e01-b9c7-4c26-99e4-fefcad87f111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:38.646809 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b06e01-b9c7-4c26-99e4-fefcad87f111 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:38.647613 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b06e01-b9c7-4c26-99e4-fefcad87f111 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:38.647910 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b06e01-b9c7-4c26-99e4-fefcad87f111 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:38.665759 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b06e01-b9c7-4c26-99e4-fefcad87f111 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:38.667309 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b06e01-b9c7-4c26-99e4-fefcad87f111 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:38.690401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-b8b06e01-b9c7-4c26-99e4-fefcad87f111 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:38.690401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:38.690401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:38.690401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:38.690401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:38.690401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:38.690401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:38.690401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:38.690401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:38.690401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:38.690401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:38.690401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:38.690401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:24:38.690401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:24:38.690401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:24:38.690401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:24:38.691448 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:38.691448 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:38.691448 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:38.691448 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:38.691448 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:38.691448 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:38.691448 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:38.691448 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:38.691448 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:38.691448 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:38.691448 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:38.691448 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:38.691448 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:38.691448 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:38.691448 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:38.691448 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:38.691448 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:24:38.692941 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:24:38.692941 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:38.692941 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:38.692941 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 272/537] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:24:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 09:24:38.699935 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc98c40-624b-43dd-aee7-07356f8bfa8f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:38.747121 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:38.789204 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d11a48-345c-4a70-9ca8-3ff68ea2db9f None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:38.789513 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d11a48-345c-4a70-9ca8-3ff68ea2db9f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:38.789740 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d11a48-345c-4a70-9ca8-3ff68ea2db9f None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:38.791317 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 273/538] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:38.796322 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2bbf28-ac52-4f99-b886-d7b96274405f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:38.796630 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2bbf28-ac52-4f99-b886-d7b96274405f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:38.796905 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2bbf28-ac52-4f99-b886-d7b96274405f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:38.820750 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-ad2bbf28-ac52-4f99-b886-d7b96274405f None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:38.821934 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ad2bbf28-ac52-4f99-b886-d7b96274405f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:38.846100 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc98c40-624b-43dd-aee7-07356f8bfa8f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:38.891909 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2479521-4a84-4c63-a246-feed07cbb73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:38.893311 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:38.893523 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2479521-4a84-4c63-a246-feed07cbb73f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:38.893977 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2479521-4a84-4c63-a246-feed07cbb73f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:38.895039 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2479521-4a84-4c63-a246-feed07cbb73f None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:38.895961 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2479521-4a84-4c63-a246-feed07cbb73f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:38.896732 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2479521-4a84-4c63-a246-feed07cbb73f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:38.899676 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2479521-4a84-4c63-a246-feed07cbb73f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:38.906427 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 274/539] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:38 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 27 09:24:38.907922 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee072047-5397-4d6b-9a67-40d0245b73b4 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:38.908262 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee072047-5397-4d6b-9a67-40d0245b73b4 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:38.908707 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee072047-5397-4d6b-9a67-40d0245b73b4 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:38.913824 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 266/541] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:38.914061 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 275/541] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:38.923465 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ef688-eb9f-49c8-85da-6b19b0393260 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:38.923465 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ef688-eb9f-49c8-85da-6b19b0393260 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:38.923465 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ef688-eb9f-49c8-85da-6b19b0393260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:38.924849 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2479521-4a84-4c63-a246-feed07cbb73f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:38.939474 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-ff2ef688-eb9f-49c8-85da-6b19b0393260 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:38.940680 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ff2ef688-eb9f-49c8-85da-6b19b0393260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:39.066753 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 276/542] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:38 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 27 09:24:39.082995 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:39.092758 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2479521-4a84-4c63-a246-feed07cbb73f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:39.128809 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0512699-16da-4c2f-a424-dfaf8e938af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:39.129362 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0512699-16da-4c2f-a424-dfaf8e938af3 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:39.129641 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0512699-16da-4c2f-a424-dfaf8e938af3 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:39.129909 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0512699-16da-4c2f-a424-dfaf8e938af3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:39.138457 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0512699-16da-4c2f-a424-dfaf8e938af3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:39.139914 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0512699-16da-4c2f-a424-dfaf8e938af3 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:39.151226 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-a0512699-16da-4c2f-a424-dfaf8e938af3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 27 09:24:39.151226 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:39.151226 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:39.151226 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:39.151226 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:39.151226 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:39.151226 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:39.151226 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:39.151226 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:39.151226 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:39.151226 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:39.151226 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:39.151226 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 09:24:39.151226 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 09:24:39.151226 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 09:24:39.151226 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 09:24:39.152168 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:39.152168 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:39.152168 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 09:24:39.152168 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:24:39.152168 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 09:24:39.152168 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:24:39.152168 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:39.152168 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:39.152168 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:39.152168 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:39.152168 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:39.152168 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:39.152168 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:39.152168 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:39.152168 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:39.152168 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:39.152168 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:39.152168 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:39.153164 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:39.153164 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:39.153164 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 09:24:39.153164 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 09:24:39.153164 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 09:24:39.153164 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 09:24:39.153164 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 09:24:39.153164 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 09:24:39.153164 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 27 09:24:39.153164 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:39.153164 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 267/543] 10.222.0.70 () {56 vars in 1093 bytes} [Thu Oct 27 09:24:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:39.158056 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ef688-eb9f-49c8-85da-6b19b0393260 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:39.232546 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0512699-16da-4c2f-a424-dfaf8e938af3 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:39.288693 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ef688-eb9f-49c8-85da-6b19b0393260 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:39.315726 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:39.328215 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaae356-f87b-4401-8cd8-c96f24e7195b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:39.329159 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaae356-f87b-4401-8cd8-c96f24e7195b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:39.330473 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaae356-f87b-4401-8cd8-c96f24e7195b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:39.330956 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaae356-f87b-4401-8cd8-c96f24e7195b None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:39.331904 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaae356-f87b-4401-8cd8-c96f24e7195b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:39.332413 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaae356-f87b-4401-8cd8-c96f24e7195b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:39.334060 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaae356-f87b-4401-8cd8-c96f24e7195b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:39.354502 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 277/544] 10.222.0.70 () {56 vars in 1133 bytes} [Thu Oct 27 09:24:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:39.359633 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaae356-f87b-4401-8cd8-c96f24e7195b None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:39.439657 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0512699-16da-4c2f-a424-dfaf8e938af3 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:39.478680 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6352d7c6-bee1-4907-bf5e-fbe6456b3629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:39.479318 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352d7c6-bee1-4907-bf5e-fbe6456b3629 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:39.479536 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352d7c6-bee1-4907-bf5e-fbe6456b3629 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:39.479801 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352d7c6-bee1-4907-bf5e-fbe6456b3629 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:39.480456 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6352d7c6-bee1-4907-bf5e-fbe6456b3629 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:39.482332 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6352d7c6-bee1-4907-bf5e-fbe6456b3629 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:39.497659 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaae356-f87b-4401-8cd8-c96f24e7195b None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:39.500060 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 268/545] 10.222.0.70 () {60 vars in 1141 bytes} [Thu Oct 27 09:24:39 2022] POST /identity/v3/services => generated 240 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:24:39.501205 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6352d7c6-bee1-4907-bf5e-fbe6456b3629 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:39.518928 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:39.558941 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985652c5-2502-416b-908a-f54e0e038faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:39.559984 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985652c5-2502-416b-908a-f54e0e038faa None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:39.560366 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985652c5-2502-416b-908a-f54e0e038faa None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:39.560771 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985652c5-2502-416b-908a-f54e0e038faa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:39.626906 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6352d7c6-bee1-4907-bf5e-fbe6456b3629 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:39.627697 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:39.662075 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985652c5-2502-416b-908a-f54e0e038faa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:39.664201 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985652c5-2502-416b-908a-f54e0e038faa None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:39.670103 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712492a0-3696-4cc1-985e-85ec7d026c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:39.670971 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712492a0-3696-4cc1-985e-85ec7d026c72 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:39.671482 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712492a0-3696-4cc1-985e-85ec7d026c72 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:39.671909 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712492a0-3696-4cc1-985e-85ec7d026c72 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:39.684697 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712492a0-3696-4cc1-985e-85ec7d026c72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:24:39.690887 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-712492a0-3696-4cc1-985e-85ec7d026c72 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 09:24:39.690887 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:39.690887 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:39.690887 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:39.690887 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:39.690887 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:39.690887 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:39.690887 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:39.690887 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:39.690887 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:39.690887 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:39.690887 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:39.690887 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:24:39.690887 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:24:39.690887 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:24:39.690887 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:24:39.692077 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:24:39.692077 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:24:39.692077 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:24:39.692077 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:24:39.692077 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:39.692077 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:39.692077 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:39.692077 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:39.692077 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:39.692077 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:39.692077 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:39.692077 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:39.692077 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:39.692077 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:39.692077 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:39.692077 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:39.693025 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:39.693025 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:39.693025 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:39.693025 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:39.693025 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:24:39.693025 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:24:39.693025 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:24:39.693025 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:24:39.693025 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:24:39.693025 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:24:39.693025 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 09:24:39.693025 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:39.701179 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 269/546] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Oct 27 09:24:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:39.703080 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0719bfc7-34d5-469e-b8ee-9a3293db5384 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:39.703515 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0719bfc7-34d5-469e-b8ee-9a3293db5384 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:39.703999 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0719bfc7-34d5-469e-b8ee-9a3293db5384 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:39.710661 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 270/547] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:39.711994 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0719bfc7-34d5-469e-b8ee-9a3293db5384 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:39.799183 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:39.809251 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 278/548] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Oct 27 09:24:39 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 27 09:24:39.809251 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56000fd2-a0db-430e-a14f-216e351d2954 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:39.809251 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56000fd2-a0db-430e-a14f-216e351d2954 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:39.809251 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56000fd2-a0db-430e-a14f-216e351d2954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:39.826533 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-56000fd2-a0db-430e-a14f-216e351d2954 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:39.827737 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-56000fd2-a0db-430e-a14f-216e351d2954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:39.839536 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0719bfc7-34d5-469e-b8ee-9a3293db5384 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:39.875874 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05299135-6a31-458b-8677-f2948191c1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:39.876482 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05299135-6a31-458b-8677-f2948191c1e6 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:39.876819 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05299135-6a31-458b-8677-f2948191c1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:39.877092 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05299135-6a31-458b-8677-f2948191c1e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:39.877745 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05299135-6a31-458b-8677-f2948191c1e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:39.878050 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05299135-6a31-458b-8677-f2948191c1e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:39.879426 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05299135-6a31-458b-8677-f2948191c1e6 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:39.888573 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 271/549] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Oct 27 09:24:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:39.892544 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05299135-6a31-458b-8677-f2948191c1e6 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:39.916343 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 279/550] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 09:24:39.917261 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56000fd2-a0db-430e-a14f-216e351d2954 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:39.935985 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:40.078758 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05299135-6a31-458b-8677-f2948191c1e6 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:40.092490 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56000fd2-a0db-430e-a14f-216e351d2954 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:40.144249 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ec8016-e39a-47dc-929f-d1e27eb72d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:40.144930 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec8016-e39a-47dc-929f-d1e27eb72d01 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:40.145209 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec8016-e39a-47dc-929f-d1e27eb72d01 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:40.145479 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec8016-e39a-47dc-929f-d1e27eb72d01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:40.148739 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:40.158648 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9dbf4b-8489-4aee-b048-ab0ccc1a7999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:40.159256 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9dbf4b-8489-4aee-b048-ab0ccc1a7999 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:40.161540 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9dbf4b-8489-4aee-b048-ab0ccc1a7999 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:40.161540 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9dbf4b-8489-4aee-b048-ab0ccc1a7999 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:40.165517 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ec8016-e39a-47dc-929f-d1e27eb72d01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:40.166981 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ec8016-e39a-47dc-929f-d1e27eb72d01 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:40.180551 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9dbf4b-8489-4aee-b048-ab0ccc1a7999 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:40.182003 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9dbf4b-8489-4aee-b048-ab0ccc1a7999 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:40.196425 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-17ec8016-e39a-47dc-929f-d1e27eb72d01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:40.196425 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:40.196425 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:40.196425 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:40.196425 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:40.196425 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:40.196425 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:40.196425 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:40.196425 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:40.196425 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:40.196425 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:40.196425 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:40.196425 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:24:40.196425 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:24:40.196425 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:24:40.196425 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:24:40.197363 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:40.197363 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:40.197363 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:40.197363 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:40.197363 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:40.197363 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:40.197363 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:40.197363 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:40.197363 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:40.197363 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:40.197363 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:40.197363 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:40.197363 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:40.197363 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:40.197363 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:40.197363 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:40.197363 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:24:40.198249 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:24:40.198249 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:40.198249 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:40.199272 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 272/551] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:24:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:40.204461 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ec8016-e39a-47dc-929f-d1e27eb72d01 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:40.206831 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-6c9dbf4b-8489-4aee-b048-ab0ccc1a7999 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:40.206831 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:40.206831 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:40.206831 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:40.206831 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:40.206831 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:40.206831 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:40.206831 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:40.206831 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:40.206831 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:40.206831 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:40.206831 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:40.206831 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:24:40.206831 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:24:40.206831 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:24:40.206831 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:24:40.208046 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:40.208046 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:40.208046 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:40.208046 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:40.208046 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:40.208046 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:40.208046 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:40.208046 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:40.208046 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:40.208046 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:40.208046 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:40.208046 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:40.208046 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:40.208046 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:40.208046 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:40.208046 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:40.208046 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:24:40.209163 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:24:40.209163 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:40.209163 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:40.211318 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 280/552] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:24:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:40.216807 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9dbf4b-8489-4aee-b048-ab0ccc1a7999 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:40.297282 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:40.347967 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9dbf4b-8489-4aee-b048-ab0ccc1a7999 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:40.356230 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ec8016-e39a-47dc-929f-d1e27eb72d01 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:40.391931 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9af1b66-271a-4f4e-a9be-27305041458f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:40.392680 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af1b66-271a-4f4e-a9be-27305041458f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:40.392964 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af1b66-271a-4f4e-a9be-27305041458f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:40.393237 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af1b66-271a-4f4e-a9be-27305041458f None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:40.393880 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9af1b66-271a-4f4e-a9be-27305041458f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:40.394184 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9af1b66-271a-4f4e-a9be-27305041458f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:40.394710 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b937cd-7d5b-40c1-afb1-b7bd01870493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:40.395579 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b937cd-7d5b-40c1-afb1-b7bd01870493 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:40.395687 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9af1b66-271a-4f4e-a9be-27305041458f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:40.396208 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b937cd-7d5b-40c1-afb1-b7bd01870493 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:40.396716 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b937cd-7d5b-40c1-afb1-b7bd01870493 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:40.398313 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b937cd-7d5b-40c1-afb1-b7bd01870493 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:40.399339 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b937cd-7d5b-40c1-afb1-b7bd01870493 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:40.400973 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b937cd-7d5b-40c1-afb1-b7bd01870493 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:40.412293 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 273/553] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:24:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:24:40.413413 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9af1b66-271a-4f4e-a9be-27305041458f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:40.418264 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 281/554] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:24:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:40.419396 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b937cd-7d5b-40c1-afb1-b7bd01870493 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:40.472120 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:40.528917 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9af1b66-271a-4f4e-a9be-27305041458f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:40.558466 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b937cd-7d5b-40c1-afb1-b7bd01870493 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:40.577234 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c59b10-915e-4338-aeb2-a9d6867fb504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:40.577882 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c59b10-915e-4338-aeb2-a9d6867fb504 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:40.578152 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c59b10-915e-4338-aeb2-a9d6867fb504 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:40.578414 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c59b10-915e-4338-aeb2-a9d6867fb504 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:40.587080 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:40.600479 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c59b10-915e-4338-aeb2-a9d6867fb504 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:24:40.603983 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7670aac-fbb3-4e0a-b1ea-b957f7e38e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:40.604722 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7670aac-fbb3-4e0a-b1ea-b957f7e38e7d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:40.605066 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7670aac-fbb3-4e0a-b1ea-b957f7e38e7d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:40.605403 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7670aac-fbb3-4e0a-b1ea-b957f7e38e7d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:40.607619 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-f8c59b10-915e-4338-aeb2-a9d6867fb504 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 27 09:24:40.607619 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:40.607619 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:40.607619 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:40.607619 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:40.607619 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:40.607619 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:40.607619 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:40.607619 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:40.607619 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:40.607619 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:40.607619 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:40.607619 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:24:40.607619 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:24:40.607619 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:24:40.607619 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:24:40.608567 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:24:40.608567 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:24:40.608567 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:24:40.608567 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:24:40.608567 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:40.608567 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:40.608567 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:40.608567 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:40.608567 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:40.608567 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:40.608567 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:40.608567 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:40.608567 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:40.608567 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:40.608567 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:40.608567 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:40.609429 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:40.609429 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:40.609429 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:40.609429 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:40.609429 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:24:40.609429 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:24:40.609429 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:24:40.609429 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:24:40.609429 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:24:40.609429 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:24:40.609429 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 27 09:24:40.609429 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:40.612581 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 274/555] 10.222.0.70 () {56 vars in 1102 bytes} [Thu Oct 27 09:24:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:40.613619 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c59b10-915e-4338-aeb2-a9d6867fb504 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:40.620368 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7670aac-fbb3-4e0a-b1ea-b957f7e38e7d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:40.621896 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7670aac-fbb3-4e0a-b1ea-b957f7e38e7d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:40.635305 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-e7670aac-fbb3-4e0a-b1ea-b957f7e38e7d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:24:40.635305 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:40.635305 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:40.635305 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:40.635305 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:40.635305 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:40.635305 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:40.635305 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:40.635305 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:40.635305 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:40.635305 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:40.635305 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:40.635305 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 09:24:40.635305 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 09:24:40.635305 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 09:24:40.636209 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 09:24:40.636209 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:40.636209 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:40.636209 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:40.636209 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:40.636209 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:40.636209 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:40.636209 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:40.636209 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:40.636209 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:40.636209 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:40.636209 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:40.636209 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:40.636209 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:40.636209 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:40.636209 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:40.636209 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:40.637463 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 09:24:40.637463 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 09:24:40.637463 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 09:24:40.637463 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 09:24:40.637463 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 09:24:40.637463 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 09:24:40.637463 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:24:40.637463 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:40.641893 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 282/556] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Oct 27 09:24:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:40.642738 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7670aac-fbb3-4e0a-b1ea-b957f7e38e7d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:40.751258 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c59b10-915e-4338-aeb2-a9d6867fb504 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:40.752473 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7670aac-fbb3-4e0a-b1ea-b957f7e38e7d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:40.761492 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:40.792232 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50545a9f-c203-4b45-ad34-9d05f0d3a75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:40.792934 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50545a9f-c203-4b45-ad34-9d05f0d3a75d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:40.793215 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50545a9f-c203-4b45-ad34-9d05f0d3a75d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:40.793482 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50545a9f-c203-4b45-ad34-9d05f0d3a75d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:40.800683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50545a9f-c203-4b45-ad34-9d05f0d3a75d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:40.801136 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82751a4b-0e21-4bd1-b1b0-1882ebbb6507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:40.801964 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82751a4b-0e21-4bd1-b1b0-1882ebbb6507 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:40.802195 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50545a9f-c203-4b45-ad34-9d05f0d3a75d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:40.802819 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82751a4b-0e21-4bd1-b1b0-1882ebbb6507 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:40.803346 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82751a4b-0e21-4bd1-b1b0-1882ebbb6507 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:40.804469 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82751a4b-0e21-4bd1-b1b0-1882ebbb6507 None admin] RBAC: Adding query filter params (name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:40.805599 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82751a4b-0e21-4bd1-b1b0-1882ebbb6507 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:40.807170 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82751a4b-0e21-4bd1-b1b0-1882ebbb6507 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:40.814358 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-50545a9f-c203-4b45-ad34-9d05f0d3a75d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 27 09:24:40.814358 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:40.814358 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:40.814358 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:40.814358 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:40.814358 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:40.814358 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:40.814358 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:40.814358 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:40.814358 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:40.814358 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:40.814358 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:40.814358 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 09:24:40.814358 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 09:24:40.814358 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 09:24:40.814358 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 09:24:40.815327 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:40.815327 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:40.815327 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 09:24:40.815327 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:24:40.815327 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 09:24:40.815327 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:24:40.815327 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:40.815327 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:40.815327 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:40.815327 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:40.815327 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:40.815327 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:40.815327 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:40.815327 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:40.815327 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:40.815327 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:40.815327 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:40.815327 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:40.816346 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:40.816346 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:40.816346 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 09:24:40.816346 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 09:24:40.816346 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 09:24:40.816346 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 09:24:40.816346 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 09:24:40.816346 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 09:24:40.816346 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 27 09:24:40.816346 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:40.819495 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 283/558] 10.222.0.70 () {56 vars in 1103 bytes} [Thu Oct 27 09:24:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:40.819880 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 275/558] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Oct 27 09:24:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:40.820773 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82751a4b-0e21-4bd1-b1b0-1882ebbb6507 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:40.821619 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50545a9f-c203-4b45-ad34-9d05f0d3a75d None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:40.884561 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:40.937171 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50545a9f-c203-4b45-ad34-9d05f0d3a75d None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:40.969916 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82751a4b-0e21-4bd1-b1b0-1882ebbb6507 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:40.975722 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54804e7d-044a-4da3-b33d-7272407293d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:40.976446 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54804e7d-044a-4da3-b33d-7272407293d6 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:40.976863 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54804e7d-044a-4da3-b33d-7272407293d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:40.977230 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54804e7d-044a-4da3-b33d-7272407293d6 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:40.978051 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54804e7d-044a-4da3-b33d-7272407293d6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:40.978452 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54804e7d-044a-4da3-b33d-7272407293d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:40.980088 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54804e7d-044a-4da3-b33d-7272407293d6 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:41.004321 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 276/559] 10.222.0.70 () {56 vars in 1135 bytes} [Thu Oct 27 09:24:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:41.005434 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54804e7d-044a-4da3-b33d-7272407293d6 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:41.013307 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685c10e6-b9b9-46d8-828e-38be676747c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:41.014125 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c10e6-b9b9-46d8-828e-38be676747c5 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:41.014575 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c10e6-b9b9-46d8-828e-38be676747c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:41.015004 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c10e6-b9b9-46d8-828e-38be676747c5 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:41.015832 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685c10e6-b9b9-46d8-828e-38be676747c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:41.016695 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685c10e6-b9b9-46d8-828e-38be676747c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:41.018119 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685c10e6-b9b9-46d8-828e-38be676747c5 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:41.031737 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 284/560] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Oct 27 09:24:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:41.032808 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685c10e6-b9b9-46d8-828e-38be676747c5 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:41.042294 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:41.119707 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54804e7d-044a-4da3-b33d-7272407293d6 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:41.139766 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685c10e6-b9b9-46d8-828e-38be676747c5 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:41.141241 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:41.157013 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57978031-a10d-48ce-8f3e-48b77f9f24e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:41.157642 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57978031-a10d-48ce-8f3e-48b77f9f24e6 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:41.157921 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57978031-a10d-48ce-8f3e-48b77f9f24e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:41.158246 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57978031-a10d-48ce-8f3e-48b77f9f24e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:41.168455 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57978031-a10d-48ce-8f3e-48b77f9f24e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:41.169977 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57978031-a10d-48ce-8f3e-48b77f9f24e6 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:41.180881 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d212b804-0059-4fdd-8567-982843f1005c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:41.181569 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d212b804-0059-4fdd-8567-982843f1005c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:41.181850 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d212b804-0059-4fdd-8567-982843f1005c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:41.182121 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d212b804-0059-4fdd-8567-982843f1005c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:41.185502 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-57978031-a10d-48ce-8f3e-48b77f9f24e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:41.185502 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:41.185502 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:41.185502 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:41.185502 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:41.185502 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:41.185502 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:41.185502 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:41.185502 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:41.185502 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:41.185502 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:41.185502 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:41.185502 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:24:41.185502 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:24:41.185502 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:24:41.185502 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:24:41.186414 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:41.186414 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:41.186414 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:41.186414 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:41.186414 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:41.186414 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:41.186414 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:41.186414 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:41.186414 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:41.186414 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:41.186414 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:41.186414 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:41.186414 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:41.186414 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:41.186414 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:41.186414 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:41.186414 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:24:41.187305 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:24:41.187305 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:41.187305 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:41.191281 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 277/561] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:24:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:41.192296 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baad985-c44a-42c9-8d6e-0e139c02bd53 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:41.192600 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baad985-c44a-42c9-8d6e-0e139c02bd53 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:41.192781 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d212b804-0059-4fdd-8567-982843f1005c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:41.192877 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baad985-c44a-42c9-8d6e-0e139c02bd53 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:41.194696 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d212b804-0059-4fdd-8567-982843f1005c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:41.198829 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 278/562] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:41.199574 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baad985-c44a-42c9-8d6e-0e139c02bd53 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:41.210851 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-d212b804-0059-4fdd-8567-982843f1005c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:41.210851 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:41.210851 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:41.210851 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:41.210851 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:41.210851 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:41.210851 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:41.210851 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:41.210851 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:41.210851 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:41.210851 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:41.210851 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:41.210851 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:24:41.210851 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:24:41.210851 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:24:41.210851 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:24:41.211839 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:41.211839 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:41.211839 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:41.211839 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:41.211839 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:41.211839 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:41.211839 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:41.211839 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:41.211839 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:41.211839 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:41.211839 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:41.211839 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:41.211839 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:41.211839 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:41.211839 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:41.211839 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:41.211839 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:24:41.212804 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:24:41.212804 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:41.212804 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:41.212804 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 285/563] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:24:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:41.212804 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d212b804-0059-4fdd-8567-982843f1005c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:41.294402 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:41.320508 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d212b804-0059-4fdd-8567-982843f1005c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:41.321987 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baad985-c44a-42c9-8d6e-0e139c02bd53 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:41.357394 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb042c5-82ed-46a2-8114-0d425cd847e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:41.357833 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c551be-6d57-48f2-abd0-b1997e8aaf2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:41.357999 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb042c5-82ed-46a2-8114-0d425cd847e2 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:41.358277 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb042c5-82ed-46a2-8114-0d425cd847e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:41.358546 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb042c5-82ed-46a2-8114-0d425cd847e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:41.358789 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c551be-6d57-48f2-abd0-b1997e8aaf2b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:41.359161 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c551be-6d57-48f2-abd0-b1997e8aaf2b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:41.359266 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb042c5-82ed-46a2-8114-0d425cd847e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:41.359458 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb042c5-82ed-46a2-8114-0d425cd847e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:41.359696 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c551be-6d57-48f2-abd0-b1997e8aaf2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:41.360497 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c551be-6d57-48f2-abd0-b1997e8aaf2b None admin] RBAC: Adding query filter params (role.id=0dfd839b7a194aee96f82f4ebc1620a7, scope.project.id=16ac205ec6f241d69d5e6b8408f09f38, user.id=dbe4a6e4b6184826b30ac4fc13ad9b1f) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:41.360851 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb042c5-82ed-46a2-8114-0d425cd847e2 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:41.360995 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c551be-6d57-48f2-abd0-b1997e8aaf2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:41.362492 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c551be-6d57-48f2-abd0-b1997e8aaf2b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:41.377397 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 279/564] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Oct 27 09:24:41 2022] GET /identity/v3/role_assignments?user.id=dbe4a6e4b6184826b30ac4fc13ad9b1f&scope.project.id=16ac205ec6f241d69d5e6b8408f09f38&role.id=0dfd839b7a194aee96f82f4ebc1620a7 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:41.377397 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36471e9-14c4-41fc-bd55-58f4d115a492 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:41.377397 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36471e9-14c4-41fc-bd55-58f4d115a492 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:41.377397 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36471e9-14c4-41fc-bd55-58f4d115a492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:41.388764 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 286/565] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:24:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:41.391340 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb042c5-82ed-46a2-8114-0d425cd847e2 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:41.396054 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-e36471e9-14c4-41fc-bd55-58f4d115a492 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:41.397971 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e36471e9-14c4-41fc-bd55-58f4d115a492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:41.420913 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:41.508958 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 280/566] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 09:24:41.510462 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36471e9-14c4-41fc-bd55-58f4d115a492 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:41.535583 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb042c5-82ed-46a2-8114-0d425cd847e2 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:41.585593 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9986a44c-b215-4ecf-8566-07da8a469f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:41.586250 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986a44c-b215-4ecf-8566-07da8a469f6a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:41.586478 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986a44c-b215-4ecf-8566-07da8a469f6a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:41.586730 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986a44c-b215-4ecf-8566-07da8a469f6a None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:41.587661 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9986a44c-b215-4ecf-8566-07da8a469f6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:41.587970 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9986a44c-b215-4ecf-8566-07da8a469f6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:41.589491 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9986a44c-b215-4ecf-8566-07da8a469f6a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:41.610289 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 287/567] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:24:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:41.612091 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9986a44c-b215-4ecf-8566-07da8a469f6a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:41.622403 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:41.669299 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36471e9-14c4-41fc-bd55-58f4d115a492 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:41.708737 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:41.722336 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94708d70-aa24-47f1-b22c-1c8584385d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:41.723697 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94708d70-aa24-47f1-b22c-1c8584385d5e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:41.724218 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94708d70-aa24-47f1-b22c-1c8584385d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:41.724712 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94708d70-aa24-47f1-b22c-1c8584385d5e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:41.734954 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94708d70-aa24-47f1-b22c-1c8584385d5e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:41.737008 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94708d70-aa24-47f1-b22c-1c8584385d5e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:41.741150 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9986a44c-b215-4ecf-8566-07da8a469f6a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:41.752364 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-94708d70-aa24-47f1-b22c-1c8584385d5e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 27 09:24:41.752364 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:41.752364 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:41.752364 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:41.752364 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:41.752364 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:41.752364 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:41.752364 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:41.752364 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:41.752364 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:41.752364 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:41.752364 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:41.752364 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 09:24:41.752364 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 09:24:41.752364 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 09:24:41.752364 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 09:24:41.753848 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:41.753848 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:41.753848 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 09:24:41.753848 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:24:41.753848 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 09:24:41.753848 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:24:41.753848 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:41.753848 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:41.753848 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:41.753848 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:41.753848 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:41.753848 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:41.753848 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:41.753848 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:41.753848 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:41.753848 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:41.753848 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:41.753848 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:41.754767 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:41.754767 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:41.754767 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 09:24:41.754767 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 09:24:41.754767 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 09:24:41.754767 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 09:24:41.754767 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 09:24:41.754767 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 09:24:41.754767 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 27 09:24:41.754767 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:41.758676 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 281/568] 10.222.0.70 () {56 vars in 1117 bytes} [Thu Oct 27 09:24:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:41.765138 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94708d70-aa24-47f1-b22c-1c8584385d5e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:41.792861 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09aa4a0-0ef6-4943-ad3e-8e58b3a53d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:41.793515 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09aa4a0-0ef6-4943-ad3e-8e58b3a53d54 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:41.793793 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09aa4a0-0ef6-4943-ad3e-8e58b3a53d54 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:41.794060 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09aa4a0-0ef6-4943-ad3e-8e58b3a53d54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:41.803329 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09aa4a0-0ef6-4943-ad3e-8e58b3a53d54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:41.812581 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09aa4a0-0ef6-4943-ad3e-8e58b3a53d54 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:41.819666 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-b09aa4a0-0ef6-4943-ad3e-8e58b3a53d54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:24:41.819666 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:41.819666 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:41.819666 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:41.819666 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:41.819666 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:41.819666 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:41.819666 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:41.819666 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:41.819666 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:41.819666 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:41.819666 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:41.819666 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 09:24:41.819666 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 09:24:41.819666 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 09:24:41.820559 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 09:24:41.820559 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:41.820559 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:41.820559 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:41.820559 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:41.820559 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:41.820559 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:41.820559 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:41.820559 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:41.820559 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:41.820559 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:41.820559 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:41.820559 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:41.820559 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:41.820559 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:41.820559 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:41.820559 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:41.821929 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 09:24:41.821929 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 09:24:41.821929 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 09:24:41.821929 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 09:24:41.821929 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 09:24:41.821929 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 09:24:41.821929 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:24:41.821929 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:41.821929 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 288/569] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Oct 27 09:24:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:41.821929 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09aa4a0-0ef6-4943-ad3e-8e58b3a53d54 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:41.917709 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94708d70-aa24-47f1-b22c-1c8584385d5e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:41.933796 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09aa4a0-0ef6-4943-ad3e-8e58b3a53d54 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:41.948684 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:41.955894 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d612a5b-ece7-4002-bc24-02ef3f69dcd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:41.956638 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d612a5b-ece7-4002-bc24-02ef3f69dcd6 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:41.956921 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d612a5b-ece7-4002-bc24-02ef3f69dcd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:41.957312 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d612a5b-ece7-4002-bc24-02ef3f69dcd6 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:41.958049 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d612a5b-ece7-4002-bc24-02ef3f69dcd6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:41.958466 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d612a5b-ece7-4002-bc24-02ef3f69dcd6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:41.960711 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d612a5b-ece7-4002-bc24-02ef3f69dcd6 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:41.968577 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d41be-20ab-4776-9a85-17004552e5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:41.969521 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d41be-20ab-4776-9a85-17004552e5e8 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:41.969865 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d41be-20ab-4776-9a85-17004552e5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:41.970201 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d41be-20ab-4776-9a85-17004552e5e8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:41.970846 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01d41be-20ab-4776-9a85-17004552e5e8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:41.972264 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01d41be-20ab-4776-9a85-17004552e5e8 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:41.974212 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:41.981605 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 282/570] 10.222.0.70 () {56 vars in 1149 bytes} [Thu Oct 27 09:24:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:41.982674 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d612a5b-ece7-4002-bc24-02ef3f69dcd6 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:41.983745 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-b01d41be-20ab-4776-9a85-17004552e5e8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 27 09:24:41.983745 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:41.983745 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 09:24:41.983745 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 09:24:41.983745 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 09:24:41.983745 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 09:24:41.983745 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 09:24:41.983745 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:24:41.983745 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 27 09:24:41.983745 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:41.983745 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 09:24:41.983745 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:41.983745 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:41.983745 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:41.983745 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:41.983745 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:41.983745 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:41.984758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:41.984758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:41.984758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:41.984758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:41.984758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:41.984758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:41.984758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 09:24:41.984758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 09:24:41.984758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 09:24:41.984758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 09:24:41.984758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:41.984758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:41.984758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:41.984758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:41.984758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:41.984758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:41.985667 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:41.985667 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:41.985667 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:41.985667 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:41.985667 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:41.985667 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:41.985667 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:41.985667 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:41.985667 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:41.985667 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:41.985667 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 09:24:41.985667 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:24:41.985667 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 27 09:24:41.985667 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:41.985667 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 289/571] 10.222.0.70 () {56 vars in 1129 bytes} [Thu Oct 27 09:24:41 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:41.986664 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d120c153-8f5b-4cd9-a903-3ee17dbbfebc None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:41.986963 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d120c153-8f5b-4cd9-a903-3ee17dbbfebc None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:41.987236 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d120c153-8f5b-4cd9-a903-3ee17dbbfebc None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:41.992002 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 290/572] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:41.993048 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d120c153-8f5b-4cd9-a903-3ee17dbbfebc None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:42.094670 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d612a5b-ece7-4002-bc24-02ef3f69dcd6 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:42.100568 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d120c153-8f5b-4cd9-a903-3ee17dbbfebc None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:42.132738 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cdc6e4-ef06-46e8-80f0-6df22f5074b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:42.133438 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cdc6e4-ef06-46e8-80f0-6df22f5074b5 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:42.133784 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cdc6e4-ef06-46e8-80f0-6df22f5074b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:42.134222 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cdc6e4-ef06-46e8-80f0-6df22f5074b5 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:42.135163 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cdc6e4-ef06-46e8-80f0-6df22f5074b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:42.135535 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cdc6e4-ef06-46e8-80f0-6df22f5074b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:42.137145 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b2c2b9-342e-4c87-b36a-33611aa4c014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:42.137326 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cdc6e4-ef06-46e8-80f0-6df22f5074b5 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:42.137903 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b2c2b9-342e-4c87-b36a-33611aa4c014 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:42.138247 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b2c2b9-342e-4c87-b36a-33611aa4c014 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:42.138584 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b2c2b9-342e-4c87-b36a-33611aa4c014 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:42.149974 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b2c2b9-342e-4c87-b36a-33611aa4c014 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:42.151639 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b2c2b9-342e-4c87-b36a-33611aa4c014 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:42.152669 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 283/573] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Oct 27 09:24:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:42.153575 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cdc6e4-ef06-46e8-80f0-6df22f5074b5 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:42.169808 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-58b2c2b9-342e-4c87-b36a-33611aa4c014 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:42.169808 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:42.169808 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:42.169808 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:42.169808 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:42.169808 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:42.169808 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:42.169808 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:42.169808 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:42.169808 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:42.169808 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:42.169808 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:42.169808 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:24:42.169808 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:24:42.169808 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:24:42.169808 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:24:42.171006 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:42.171006 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:42.171006 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:42.171006 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:42.171006 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:42.171006 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:42.171006 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:42.171006 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:42.171006 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:42.171006 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:42.171006 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:42.171006 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:42.171006 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:42.171006 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:42.171006 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:42.171006 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:42.171006 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:24:42.172095 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:24:42.172095 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:42.172095 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:42.178835 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 291/574] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:24:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:42.178835 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836bc418-6cd6-4598-a7f4-a90ff9f5d3df None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:42.178835 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836bc418-6cd6-4598-a7f4-a90ff9f5d3df None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:42.178835 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836bc418-6cd6-4598-a7f4-a90ff9f5d3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:42.196055 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-836bc418-6cd6-4598-a7f4-a90ff9f5d3df None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:42.197339 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-836bc418-6cd6-4598-a7f4-a90ff9f5d3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:42.209190 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:42.248940 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:42.263301 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cdc6e4-ef06-46e8-80f0-6df22f5074b5 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:42.275772 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 292/575] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 09:24:42.276933 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836bc418-6cd6-4598-a7f4-a90ff9f5d3df None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:42.312138 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ab8b7-fc39-48e3-9212-36c3a8bd4e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:42.313020 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ab8b7-fc39-48e3-9212-36c3a8bd4e8e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:42.313464 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ab8b7-fc39-48e3-9212-36c3a8bd4e8e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:42.313942 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ab8b7-fc39-48e3-9212-36c3a8bd4e8e None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:42.314913 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ab8b7-fc39-48e3-9212-36c3a8bd4e8e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:42.315536 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ab8b7-fc39-48e3-9212-36c3a8bd4e8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:42.316969 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ab8b7-fc39-48e3-9212-36c3a8bd4e8e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:42.327210 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 284/576] 10.222.0.70 () {56 vars in 1124 bytes} [Thu Oct 27 09:24:42 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:42.327210 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ab8b7-fc39-48e3-9212-36c3a8bd4e8e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:42.420996 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836bc418-6cd6-4598-a7f4-a90ff9f5d3df None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:42.458836 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ab8b7-fc39-48e3-9212-36c3a8bd4e8e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:42.474712 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:42.493875 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29634312-eeec-4ab9-9635-23aa0c6f59d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:42.495651 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29634312-eeec-4ab9-9635-23aa0c6f59d4 None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:42.496552 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29634312-eeec-4ab9-9635-23aa0c6f59d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:42.497360 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29634312-eeec-4ab9-9635-23aa0c6f59d4 None admin] PATH_INFO: `/v3/projects/16ac205ec6f241d69d5e6b8408f09f38/users/eecf79ea4700496c89b1a67c614b34d0/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:42.500818 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29634312-eeec-4ab9-9635-23aa0c6f59d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=16ac205ec6f241d69d5e6b8408f09f38, user_id=eecf79ea4700496c89b1a67c614b34d0, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:42.502374 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623e5394-2f80-4171-abf4-dc59d31c073d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:42.503081 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e5394-2f80-4171-abf4-dc59d31c073d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:42.503463 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e5394-2f80-4171-abf4-dc59d31c073d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:42.503850 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e5394-2f80-4171-abf4-dc59d31c073d None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:42.504614 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29634312-eeec-4ab9-9635-23aa0c6f59d4 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:42.504782 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623e5394-2f80-4171-abf4-dc59d31c073d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:42.505108 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623e5394-2f80-4171-abf4-dc59d31c073d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:42.506685 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623e5394-2f80-4171-abf4-dc59d31c073d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:42.529068 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 285/577] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:24:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:42.529068 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623e5394-2f80-4171-abf4-dc59d31c073d None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:42.537058 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 293/578] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Oct 27 09:24:42 2022] PUT /identity/v3/projects/16ac205ec6f241d69d5e6b8408f09f38/users/eecf79ea4700496c89b1a67c614b34d0/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:24:42.540656 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29634312-eeec-4ab9-9635-23aa0c6f59d4 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:42.624879 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:42.704380 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623e5394-2f80-4171-abf4-dc59d31c073d None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:42.743245 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29634312-eeec-4ab9-9635-23aa0c6f59d4 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:42.755859 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d8691-d09b-48c0-b211-e5b07b5336d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:42.757264 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d8691-d09b-48c0-b211-e5b07b5336d1 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:42.757862 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d8691-d09b-48c0-b211-e5b07b5336d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:42.758432 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d8691-d09b-48c0-b211-e5b07b5336d1 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:42.759645 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21d8691-d09b-48c0-b211-e5b07b5336d1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:42.760281 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21d8691-d09b-48c0-b211-e5b07b5336d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:42.761955 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21d8691-d09b-48c0-b211-e5b07b5336d1 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:42.772630 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 286/579] 10.222.0.70 () {56 vars in 1124 bytes} [Thu Oct 27 09:24:42 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:42.772630 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d8691-d09b-48c0-b211-e5b07b5336d1 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:42.791320 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe79617c-be12-4253-8d14-e9b02257741e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:42.792323 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe79617c-be12-4253-8d14-e9b02257741e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:42.792828 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe79617c-be12-4253-8d14-e9b02257741e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:42.793315 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe79617c-be12-4253-8d14-e9b02257741e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:42.801986 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe79617c-be12-4253-8d14-e9b02257741e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:42.804387 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe79617c-be12-4253-8d14-e9b02257741e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:42.816745 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-fe79617c-be12-4253-8d14-e9b02257741e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:24:42.816745 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:42.816745 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:42.816745 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:42.816745 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:42.816745 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:42.816745 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:42.816745 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:42.816745 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:42.816745 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:42.816745 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:42.816745 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:42.816745 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 09:24:42.816745 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 09:24:42.816745 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 09:24:42.817728 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 09:24:42.817728 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:42.817728 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:42.817728 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:42.817728 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:42.817728 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:42.817728 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:42.817728 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:42.817728 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:42.817728 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:42.817728 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:42.817728 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:42.817728 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:42.817728 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:42.817728 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:42.817728 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:42.817728 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:42.818896 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 09:24:42.818896 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 09:24:42.818896 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 09:24:42.818896 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 09:24:42.818896 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 09:24:42.818896 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 09:24:42.818896 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:24:42.818896 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:42.825881 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 294/580] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Oct 27 09:24:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:42.825881 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe79617c-be12-4253-8d14-e9b02257741e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:42.829469 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:42.885965 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d8691-d09b-48c0-b211-e5b07b5336d1 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:42.931214 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72786229-63d9-4a28-8334-ec88f430d90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:42.932388 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72786229-63d9-4a28-8334-ec88f430d90e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:42.932827 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72786229-63d9-4a28-8334-ec88f430d90e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:42.933099 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72786229-63d9-4a28-8334-ec88f430d90e None admin] PATH_INFO: `/v3/services/network` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:42.933934 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72786229-63d9-4a28-8334-ec88f430d90e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:42.935023 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:42.936667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72786229-63d9-4a28-8334-ec88f430d90e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:42.938274 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe79617c-be12-4253-8d14-e9b02257741e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:42.949805 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-72786229-63d9-4a28-8334-ec88f430d90e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 27 09:24:42.949805 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:42.949805 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 09:24:42.949805 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 09:24:42.949805 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 09:24:42.949805 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 09:24:42.949805 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 09:24:42.949805 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:24:42.949805 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 27 09:24:42.949805 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:42.949805 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 09:24:42.949805 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:42.949805 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:42.949805 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:42.949805 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:42.949805 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:42.949805 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:42.950918 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:42.950918 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:42.950918 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:42.950918 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:42.950918 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:42.950918 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:42.950918 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 09:24:42.950918 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 09:24:42.950918 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 09:24:42.950918 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 09:24:42.950918 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:42.950918 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:42.950918 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:42.950918 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:42.950918 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:42.950918 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:42.951983 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:42.951983 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:42.951983 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:42.951983 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:42.951983 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:42.951983 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:42.951983 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:42.951983 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:42.951983 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:42.951983 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:42.951983 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 09:24:42.951983 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:24:42.951983 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 27 09:24:42.951983 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:42.951983 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 287/581] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Oct 27 09:24:42 2022] GET /identity/v3/services/network => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:42.952843 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72786229-63d9-4a28-8334-ec88f430d90e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:42.986197 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f02ff-95a4-437a-a794-7072f1f22fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:42.987264 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f02ff-95a4-437a-a794-7072f1f22fa4 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:42.987549 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f02ff-95a4-437a-a794-7072f1f22fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:42.987821 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f02ff-95a4-437a-a794-7072f1f22fa4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:42.988512 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411f02ff-95a4-437a-a794-7072f1f22fa4 None admin] RBAC: Adding query filter params (interface=public, service_id=bd41834f705242da9bbfd2a1f1f3e539, region_id=RegionOne) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:42.988845 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411f02ff-95a4-437a-a794-7072f1f22fa4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:42.990769 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411f02ff-95a4-437a-a794-7072f1f22fa4 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:43.001702 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 295/582] 10.222.0.70 () {56 vars in 1251 bytes} [Thu Oct 27 09:24:42 2022] GET /identity/v3/endpoints?service_id=bd41834f705242da9bbfd2a1f1f3e539&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:43.002735 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f02ff-95a4-437a-a794-7072f1f22fa4 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:43.092372 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:43.097175 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72786229-63d9-4a28-8334-ec88f430d90e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:43.130481 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f02ff-95a4-437a-a794-7072f1f22fa4 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:43.139424 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da0692-c3af-4fd5-b601-2c8a1681133d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:43.140228 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da0692-c3af-4fd5-b601-2c8a1681133d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:43.141036 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da0692-c3af-4fd5-b601-2c8a1681133d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:43.141478 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da0692-c3af-4fd5-b601-2c8a1681133d None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:43.142629 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2da0692-c3af-4fd5-b601-2c8a1681133d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:43.143361 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2da0692-c3af-4fd5-b601-2c8a1681133d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:43.145203 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2da0692-c3af-4fd5-b601-2c8a1681133d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:43.159846 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 288/583] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Oct 27 09:24:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:43.160900 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da0692-c3af-4fd5-b601-2c8a1681133d None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:43.173125 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb3ff5-4840-498a-ab0d-dcfa50f9b05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:43.173724 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb3ff5-4840-498a-ab0d-dcfa50f9b05a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:43.173994 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb3ff5-4840-498a-ab0d-dcfa50f9b05a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:43.174255 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb3ff5-4840-498a-ab0d-dcfa50f9b05a None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:43.174870 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bb3ff5-4840-498a-ab0d-dcfa50f9b05a None admin] RBAC: Adding query filter params (name=network) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:43.175166 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bb3ff5-4840-498a-ab0d-dcfa50f9b05a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:43.176543 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bb3ff5-4840-498a-ab0d-dcfa50f9b05a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:43.185155 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 296/584] 10.222.0.70 () {56 vars in 1112 bytes} [Thu Oct 27 09:24:43 2022] GET /identity/v3/services?name=network => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:43.186493 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb3ff5-4840-498a-ab0d-dcfa50f9b05a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:43.223599 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:43.281215 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da0692-c3af-4fd5-b601-2c8a1681133d None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:43.309185 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb3ff5-4840-498a-ab0d-dcfa50f9b05a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:43.318089 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a2ff4-f0e5-4c70-8f3d-7d76de3479b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:43.319567 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a2ff4-f0e5-4c70-8f3d-7d76de3479b1 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:43.320194 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a2ff4-f0e5-4c70-8f3d-7d76de3479b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:43.320644 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a2ff4-f0e5-4c70-8f3d-7d76de3479b1 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:43.321346 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09a2ff4-f0e5-4c70-8f3d-7d76de3479b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:43.322948 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09a2ff4-f0e5-4c70-8f3d-7d76de3479b1 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:43.328968 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 289/585] 10.222.0.70 () {56 vars in 1087 bytes} [Thu Oct 27 09:24:43 2022] GET /identity/v3/services => generated 772 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:43.329846 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a2ff4-f0e5-4c70-8f3d-7d76de3479b1 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:43.364745 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4490cc0-a03d-473c-94b0-5c95a9864ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:43.365431 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4490cc0-a03d-473c-94b0-5c95a9864ad2 None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:43.365790 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4490cc0-a03d-473c-94b0-5c95a9864ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:43.366064 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4490cc0-a03d-473c-94b0-5c95a9864ad2 None admin] PATH_INFO: `/v3/projects/16ac205ec6f241d69d5e6b8408f09f38/users/7f389510a6a044d8a7487561b169544b/roles/5146b19817b54113be3f81111ef9d787` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:43.388919 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:43.402432 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4490cc0-a03d-473c-94b0-5c95a9864ad2 None admin] RBAC: Authorizing `identity:create_grant(project_id=16ac205ec6f241d69d5e6b8408f09f38, user_id=7f389510a6a044d8a7487561b169544b, role_id=5146b19817b54113be3f81111ef9d787)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:43.403937 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4490cc0-a03d-473c-94b0-5c95a9864ad2 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:43.429531 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 297/586] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Oct 27 09:24:43 2022] PUT /identity/v3/projects/16ac205ec6f241d69d5e6b8408f09f38/users/7f389510a6a044d8a7487561b169544b/roles/5146b19817b54113be3f81111ef9d787 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:24:43.507235 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a2ff4-f0e5-4c70-8f3d-7d76de3479b1 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:43.547787 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceb021a-95ad-462b-a163-023e01724769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:43.548324 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb021a-95ad-462b-a163-023e01724769 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:43.548654 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb021a-95ad-462b-a163-023e01724769 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:43.548932 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb021a-95ad-462b-a163-023e01724769 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:43.549571 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceb021a-95ad-462b-a163-023e01724769 None admin] RBAC: Adding query filter params (type=network) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:43.549873 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceb021a-95ad-462b-a163-023e01724769 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:43.551305 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceb021a-95ad-462b-a163-023e01724769 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:43.557755 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 290/587] 10.222.0.70 () {56 vars in 1112 bytes} [Thu Oct 27 09:24:43 2022] GET /identity/v3/services?type=network => generated 350 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:43.565797 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4490cc0-a03d-473c-94b0-5c95a9864ad2 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:43.660010 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:43.687327 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4490cc0-a03d-473c-94b0-5c95a9864ad2 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:43.725551 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e446c1c4-febf-4948-8292-f9e77778ff09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:43.726160 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446c1c4-febf-4948-8292-f9e77778ff09 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:43.726436 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446c1c4-febf-4948-8292-f9e77778ff09 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:43.726707 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446c1c4-febf-4948-8292-f9e77778ff09 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:43.727453 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e446c1c4-febf-4948-8292-f9e77778ff09 None admin] RBAC: Adding query filter params (interface=public, service_id=af5011040e244befa586c8b1e6d46563, region_id=RegionOne) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:43.727763 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e446c1c4-febf-4948-8292-f9e77778ff09 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:43.729147 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e446c1c4-febf-4948-8292-f9e77778ff09 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:43.734978 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 298/588] 10.222.0.70 () {56 vars in 1251 bytes} [Thu Oct 27 09:24:43 2022] GET /identity/v3/endpoints?service_id=af5011040e244befa586c8b1e6d46563&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:43.740370 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceb021a-95ad-462b-a163-023e01724769 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:43.858030 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:43.876753 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceb021a-95ad-462b-a163-023e01724769 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:43.884757 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4edec9-cdfd-4e6f-98a5-25801599bd50 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:43.885042 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4edec9-cdfd-4e6f-98a5-25801599bd50 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:43.885316 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4edec9-cdfd-4e6f-98a5-25801599bd50 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:43.886775 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 299/589] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:43.892316 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc494f9b-a56b-49ad-930f-ce0236877fe0 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:43.892316 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc494f9b-a56b-49ad-930f-ce0236877fe0 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:43.892316 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc494f9b-a56b-49ad-930f-ce0236877fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:43.921107 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-fc494f9b-a56b-49ad-930f-ce0236877fe0 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:43.922686 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-fc494f9b-a56b-49ad-930f-ce0236877fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:43.934598 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158e467e-192a-4666-b479-13909e8f80c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:43.935310 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e467e-192a-4666-b479-13909e8f80c8 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:43.935561 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e467e-192a-4666-b479-13909e8f80c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:43.935840 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e467e-192a-4666-b479-13909e8f80c8 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:43.941114 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158e467e-192a-4666-b479-13909e8f80c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:43.942385 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158e467e-192a-4666-b479-13909e8f80c8 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:43.951018 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 291/590] 10.222.0.70 () {56 vars in 1087 bytes} [Thu Oct 27 09:24:43 2022] GET /identity/v3/services => generated 772 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:24:44.001696 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 300/591] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 27 09:24:44.351582 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158e467e-192a-4666-b479-13909e8f80c8 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:44.491246 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158e467e-192a-4666-b479-13909e8f80c8 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:44.535690 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa379be-edb9-4698-8d48-fcc61218a238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:44.536241 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa379be-edb9-4698-8d48-fcc61218a238 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:44.536534 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa379be-edb9-4698-8d48-fcc61218a238 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:44.536866 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa379be-edb9-4698-8d48-fcc61218a238 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:44.537535 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa379be-edb9-4698-8d48-fcc61218a238 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:44.539386 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa379be-edb9-4698-8d48-fcc61218a238 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:44.548683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-6aa379be-edb9-4698-8d48-fcc61218a238 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 27 09:24:44.548683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:44.548683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 09:24:44.548683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 09:24:44.548683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 09:24:44.548683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 09:24:44.548683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 09:24:44.548683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:24:44.548683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 27 09:24:44.548683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:44.548683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 09:24:44.548683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:44.548683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:44.548683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:44.548683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:44.548683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:44.549626 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:44.549626 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:44.549626 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:44.549626 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:44.549626 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:44.549626 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:44.549626 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:44.549626 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 09:24:44.549626 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 09:24:44.549626 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 09:24:44.549626 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 09:24:44.549626 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:44.549626 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:44.549626 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:44.549626 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:44.549626 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:44.549626 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:44.550486 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:44.550486 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:44.550486 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:44.550486 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:44.550486 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:44.550486 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:44.550486 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:44.550486 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:44.550486 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:44.550486 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:44.550486 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 09:24:44.550486 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:24:44.550486 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 27 09:24:44.550486 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:44.550486 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 292/592] 10.222.0.70 () {56 vars in 1132 bytes} [Thu Oct 27 09:24:44 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:44.557988 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc494f9b-a56b-49ad-930f-ce0236877fe0 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:44.649821 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:44.687062 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc494f9b-a56b-49ad-930f-ce0236877fe0 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:44.733364 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e760ad02-11fa-4587-8e45-575004fe9541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:44.734034 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e760ad02-11fa-4587-8e45-575004fe9541 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:44.734312 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e760ad02-11fa-4587-8e45-575004fe9541 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:44.734633 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e760ad02-11fa-4587-8e45-575004fe9541 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:44.735259 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e760ad02-11fa-4587-8e45-575004fe9541 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:44.735561 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e760ad02-11fa-4587-8e45-575004fe9541 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:44.736957 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e760ad02-11fa-4587-8e45-575004fe9541 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:44.744672 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 301/593] 10.222.0.70 () {56 vars in 1126 bytes} [Thu Oct 27 09:24:44 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:44.750164 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa379be-edb9-4698-8d48-fcc61218a238 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:44.775874 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:44.899589 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa379be-edb9-4698-8d48-fcc61218a238 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:44.942409 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5902a4-e75f-473b-bf99-cf5f27be5865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:44.943420 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5902a4-e75f-473b-bf99-cf5f27be5865 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:44.944433 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5902a4-e75f-473b-bf99-cf5f27be5865 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:44.944918 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5902a4-e75f-473b-bf99-cf5f27be5865 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:44.945854 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5902a4-e75f-473b-bf99-cf5f27be5865 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:44.946393 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5902a4-e75f-473b-bf99-cf5f27be5865 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:44.948028 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5902a4-e75f-473b-bf99-cf5f27be5865 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:44.960921 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 293/594] 10.222.0.70 () {56 vars in 1126 bytes} [Thu Oct 27 09:24:44 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:45.071829 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c997cf6-d12b-4c62-bc3c-5c00bb15b95f None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:45.072167 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c997cf6-d12b-4c62-bc3c-5c00bb15b95f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:45.072378 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c997cf6-d12b-4c62-bc3c-5c00bb15b95f None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:45.074006 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 302/595] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:45.079913 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c35464-ed8e-4890-b7d9-13de67c7738c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:45.080243 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c35464-ed8e-4890-b7d9-13de67c7738c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:45.080452 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c35464-ed8e-4890-b7d9-13de67c7738c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:45.100506 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b6c35464-ed8e-4890-b7d9-13de67c7738c None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:45.106029 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b6c35464-ed8e-4890-b7d9-13de67c7738c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:45.209138 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 294/596] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 09:24:45.546199 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c997cf6-d12b-4c62-bc3c-5c00bb15b95f None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:45.689217 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c997cf6-d12b-4c62-bc3c-5c00bb15b95f None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:45.731548 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54ce1e0-258d-4ee5-8d51-5f1270fcd16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:45.732171 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ce1e0-258d-4ee5-8d51-5f1270fcd16b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:45.732444 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ce1e0-258d-4ee5-8d51-5f1270fcd16b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:45.732730 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ce1e0-258d-4ee5-8d51-5f1270fcd16b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:45.735464 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:45.749629 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54ce1e0-258d-4ee5-8d51-5f1270fcd16b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:24:45.754414 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-f54ce1e0-258d-4ee5-8d51-5f1270fcd16b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 09:24:45.754414 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:45.754414 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:45.754414 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:45.754414 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:45.754414 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:45.754414 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:45.754414 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:45.754414 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:45.754414 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:45.754414 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:45.754414 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:45.754414 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:24:45.754414 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:24:45.754414 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:24:45.754414 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:24:45.755324 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:24:45.755324 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:24:45.755324 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:24:45.755324 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:24:45.755324 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:45.755324 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:45.755324 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:45.755324 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:45.755324 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:45.755324 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:45.755324 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:45.755324 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:45.755324 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:45.755324 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:45.755324 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:45.755324 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:45.756195 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:45.756195 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:45.756195 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:45.756195 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:45.756195 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:24:45.756195 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:24:45.756195 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:24:45.756195 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:24:45.756195 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:24:45.756195 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:24:45.756195 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 09:24:45.756195 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:45.756195 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 303/597] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Oct 27 09:24:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:45.763266 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c35464-ed8e-4890-b7d9-13de67c7738c None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:45.780020 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:45.914478 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c35464-ed8e-4890-b7d9-13de67c7738c None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:45.948960 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4049cb44-0f3c-4ffd-8dde-0a3b5d1846a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:45.949923 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4049cb44-0f3c-4ffd-8dde-0a3b5d1846a8 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:45.950405 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4049cb44-0f3c-4ffd-8dde-0a3b5d1846a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:45.950900 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4049cb44-0f3c-4ffd-8dde-0a3b5d1846a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:45.951766 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4049cb44-0f3c-4ffd-8dde-0a3b5d1846a8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:45.952270 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4049cb44-0f3c-4ffd-8dde-0a3b5d1846a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:45.953856 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4049cb44-0f3c-4ffd-8dde-0a3b5d1846a8 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:45.963181 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 295/598] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Oct 27 09:24:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:45.969109 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54ce1e0-258d-4ee5-8d51-5f1270fcd16b None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:45.994687 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440be7d8-24e9-4c07-adfa-ae8f9960069d None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:45.995293 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440be7d8-24e9-4c07-adfa-ae8f9960069d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:45.995768 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440be7d8-24e9-4c07-adfa-ae8f9960069d None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:45.997564 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 296/599] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:46.006053 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b5fd4e-ffc0-43f7-ae4f-35e73750fe25 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:46.006391 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b5fd4e-ffc0-43f7-ae4f-35e73750fe25 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:46.006667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b5fd4e-ffc0-43f7-ae4f-35e73750fe25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:46.038202 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-32b5fd4e-ffc0-43f7-ae4f-35e73750fe25 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:46.039405 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-32b5fd4e-ffc0-43f7-ae4f-35e73750fe25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:46.112881 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54ce1e0-258d-4ee5-8d51-5f1270fcd16b None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:46.150217 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe857b-77e6-474d-9dc7-414f30664622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:46.150845 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe857b-77e6-474d-9dc7-414f30664622 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:46.151119 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe857b-77e6-474d-9dc7-414f30664622 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:46.151389 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe857b-77e6-474d-9dc7-414f30664622 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:46.161759 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbe857b-77e6-474d-9dc7-414f30664622 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:46.163815 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbe857b-77e6-474d-9dc7-414f30664622 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:46.179599 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-4cbe857b-77e6-474d-9dc7-414f30664622 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:46.179599 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:46.179599 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:46.179599 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:46.179599 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:46.179599 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:46.179599 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:46.179599 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:46.179599 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:46.179599 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:46.179599 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:46.179599 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:46.179599 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:24:46.179599 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:24:46.179599 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:24:46.179599 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:24:46.180606 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:46.180606 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:46.180606 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:46.180606 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:46.180606 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:46.180606 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:46.180606 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:46.180606 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:46.180606 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:46.180606 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:46.180606 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:46.180606 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:46.180606 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:46.180606 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:46.180606 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:46.180606 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:46.180606 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:24:46.181500 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:24:46.181500 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:46.181500 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:46.181500 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 304/600] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:24:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:46.189400 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe857b-77e6-474d-9dc7-414f30664622 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:46.222173 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 297/601] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 27 09:24:46.252474 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:46.330596 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe857b-77e6-474d-9dc7-414f30664622 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:46.384477 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae379f6-1bbf-4d70-95e7-11f64a105895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:46.385109 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae379f6-1bbf-4d70-95e7-11f64a105895 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:46.385383 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae379f6-1bbf-4d70-95e7-11f64a105895 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:46.385650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae379f6-1bbf-4d70-95e7-11f64a105895 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:46.386280 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae379f6-1bbf-4d70-95e7-11f64a105895 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:46.386578 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae379f6-1bbf-4d70-95e7-11f64a105895 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:46.387963 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae379f6-1bbf-4d70-95e7-11f64a105895 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:46.409055 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 305/602] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:24:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:46.414963 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b5fd4e-ffc0-43f7-ae4f-35e73750fe25 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:46.417622 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:46.531286 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6167a-db9e-45e1-900b-2cbafaf8e92f None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:46.531562 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6167a-db9e-45e1-900b-2cbafaf8e92f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:46.531793 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6167a-db9e-45e1-900b-2cbafaf8e92f None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:46.533347 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 306/603] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:46.542047 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617fc680-7176-425d-a384-c258571591fc None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:46.542326 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617fc680-7176-425d-a384-c258571591fc None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:46.542587 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617fc680-7176-425d-a384-c258571591fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:46.574419 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-617fc680-7176-425d-a384-c258571591fc None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:46.576145 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-617fc680-7176-425d-a384-c258571591fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:46.587176 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b5fd4e-ffc0-43f7-ae4f-35e73750fe25 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:46.633590 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3352cdab-12f3-4526-92cb-f4d31c139841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:46.634586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352cdab-12f3-4526-92cb-f4d31c139841 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:46.635049 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352cdab-12f3-4526-92cb-f4d31c139841 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:46.635519 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352cdab-12f3-4526-92cb-f4d31c139841 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:46.644945 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3352cdab-12f3-4526-92cb-f4d31c139841 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:46.646967 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3352cdab-12f3-4526-92cb-f4d31c139841 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:46.657938 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 307/604] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 27 09:24:46.658984 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617fc680-7176-425d-a384-c258571591fc None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:46.659858 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-3352cdab-12f3-4526-92cb-f4d31c139841 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 27 09:24:46.659858 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:46.659858 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:46.659858 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:46.659858 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:46.659858 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:46.659858 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:46.659858 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:46.659858 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:46.659858 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:46.659858 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:46.659858 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:46.659858 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 09:24:46.659858 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 09:24:46.659858 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 09:24:46.659858 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 09:24:46.661046 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:46.661046 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:46.661046 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 09:24:46.661046 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:24:46.661046 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 09:24:46.661046 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:24:46.661046 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:46.661046 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:46.661046 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:46.661046 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:46.661046 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:46.661046 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:46.661046 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:46.661046 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:46.661046 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:46.661046 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:46.661046 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:46.661046 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:46.662247 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:46.662247 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:46.662247 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 09:24:46.662247 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 09:24:46.662247 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 09:24:46.662247 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 09:24:46.662247 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 09:24:46.662247 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 09:24:46.662247 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 27 09:24:46.662247 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:46.664608 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 298/605] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Oct 27 09:24:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:46.677607 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3352cdab-12f3-4526-92cb-f4d31c139841 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:46.734902 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:46.814852 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:46.844082 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617fc680-7176-425d-a384-c258571591fc None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:46.891573 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3352cdab-12f3-4526-92cb-f4d31c139841 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:46.915721 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1699226a-4544-423b-8f27-b41e13c05e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:46.916360 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699226a-4544-423b-8f27-b41e13c05e25 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:46.918614 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699226a-4544-423b-8f27-b41e13c05e25 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:46.918907 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699226a-4544-423b-8f27-b41e13c05e25 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:46.935711 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1699226a-4544-423b-8f27-b41e13c05e25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:24:46.942158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-1699226a-4544-423b-8f27-b41e13c05e25 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 27 09:24:46.942158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:46.942158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:46.942158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:46.942158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:46.942158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:46.942158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:46.942158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:46.942158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:46.942158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:46.942158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:46.942158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:46.942158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:24:46.942158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:24:46.942158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:24:46.942158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:24:46.943128 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:24:46.943128 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:24:46.943128 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:24:46.943128 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:24:46.943128 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:46.943128 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:46.943128 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:46.943128 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:46.943128 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:46.943128 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:46.943128 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:46.943128 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:46.943128 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:46.943128 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:46.943128 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:46.943128 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:46.944925 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:46.944925 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:46.944925 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:46.944925 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:46.944925 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:24:46.944925 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:24:46.944925 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:24:46.944925 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:24:46.944925 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:24:46.944925 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:24:46.944925 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 27 09:24:46.944925 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:46.944925 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 308/606] 10.222.0.70 () {56 vars in 1102 bytes} [Thu Oct 27 09:24:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:46.946993 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0baa7-ed9a-4a54-9c72-2ab27681b67c None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:46.946993 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0baa7-ed9a-4a54-9c72-2ab27681b67c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:46.946993 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0baa7-ed9a-4a54-9c72-2ab27681b67c None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:46.947701 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 309/607] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:46.948423 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be0baa7-ed9a-4a54-9c72-2ab27681b67c None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:46.955818 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9626cea-8802-41a3-90c2-c7034356dff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:46.956432 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9626cea-8802-41a3-90c2-c7034356dff9 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:46.956742 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9626cea-8802-41a3-90c2-c7034356dff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:46.957022 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9626cea-8802-41a3-90c2-c7034356dff9 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:46.957743 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9626cea-8802-41a3-90c2-c7034356dff9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:46.958049 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9626cea-8802-41a3-90c2-c7034356dff9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:46.959510 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9626cea-8802-41a3-90c2-c7034356dff9 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:46.982246 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 299/608] 10.222.0.70 () {56 vars in 1135 bytes} [Thu Oct 27 09:24:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:46.982246 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9626cea-8802-41a3-90c2-c7034356dff9 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:47.060155 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:47.066095 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be0baa7-ed9a-4a54-9c72-2ab27681b67c None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:47.087090 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9626cea-8802-41a3-90c2-c7034356dff9 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:47.105812 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fceaa4-ea66-4dff-bb61-915445da05f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:47.106504 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fceaa4-ea66-4dff-bb61-915445da05f9 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:47.106786 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fceaa4-ea66-4dff-bb61-915445da05f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:47.107071 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fceaa4-ea66-4dff-bb61-915445da05f9 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:47.107674 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fceaa4-ea66-4dff-bb61-915445da05f9 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:47.108111 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:47.110247 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fceaa4-ea66-4dff-bb61-915445da05f9 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:47.121306 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-b8fceaa4-ea66-4dff-bb61-915445da05f9 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 27 09:24:47.121306 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:47.121306 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 09:24:47.121306 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 09:24:47.121306 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 09:24:47.121306 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 09:24:47.121306 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 09:24:47.121306 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:24:47.121306 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 27 09:24:47.121306 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:47.121306 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 09:24:47.121306 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:47.121306 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:47.121306 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:47.121306 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:47.121306 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:47.121306 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:47.122442 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:47.122442 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:47.122442 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:47.122442 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:47.122442 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:47.122442 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:47.122442 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 09:24:47.122442 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 09:24:47.122442 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 09:24:47.122442 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 09:24:47.122442 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:47.122442 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:47.122442 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:47.122442 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:47.122442 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:47.122442 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:47.123295 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:47.123295 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:47.123295 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:47.123295 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:47.123295 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:47.123295 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:47.123295 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:47.123295 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:47.123295 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:47.123295 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:47.123295 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 09:24:47.123295 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:24:47.123295 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 27 09:24:47.123295 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:47.123295 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 310/609] 10.222.0.70 () {56 vars in 1129 bytes} [Thu Oct 27 09:24:46 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:47.124099 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b0a168-5128-46a6-902b-64774fac8eac None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:47.124099 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b0a168-5128-46a6-902b-64774fac8eac None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:47.124229 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b0a168-5128-46a6-902b-64774fac8eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:47.129268 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29024d39-0927-4754-96ca-f0d317e19596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:47.129878 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29024d39-0927-4754-96ca-f0d317e19596 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:47.130160 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29024d39-0927-4754-96ca-f0d317e19596 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:47.130426 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29024d39-0927-4754-96ca-f0d317e19596 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:47.131121 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29024d39-0927-4754-96ca-f0d317e19596 None admin] RBAC: Adding query filter params (name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:47.131426 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29024d39-0927-4754-96ca-f0d317e19596 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:47.132902 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29024d39-0927-4754-96ca-f0d317e19596 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:47.146207 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 300/610] 10.222.0.70 () {56 vars in 1103 bytes} [Thu Oct 27 09:24:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:47.146207 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29024d39-0927-4754-96ca-f0d317e19596 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:47.146450 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-b3b0a168-5128-46a6-902b-64774fac8eac None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:47.146450 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b3b0a168-5128-46a6-902b-64774fac8eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:47.223127 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 311/611] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 09:24:47.223853 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b0a168-5128-46a6-902b-64774fac8eac None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:47.256938 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29024d39-0927-4754-96ca-f0d317e19596 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:47.303546 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc493ac8-786e-431d-8407-2ac95dd81556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:47.304498 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc493ac8-786e-431d-8407-2ac95dd81556 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:47.305034 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc493ac8-786e-431d-8407-2ac95dd81556 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:47.305508 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc493ac8-786e-431d-8407-2ac95dd81556 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:47.341179 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:47.352439 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b0a168-5128-46a6-902b-64774fac8eac None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:47.359450 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:47.379942 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc493ac8-786e-431d-8407-2ac95dd81556 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:47.381810 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc493ac8-786e-431d-8407-2ac95dd81556 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:47.388802 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91304c0d-a921-4239-80da-fdfedde5ec26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:47.389462 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91304c0d-a921-4239-80da-fdfedde5ec26 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:47.389740 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91304c0d-a921-4239-80da-fdfedde5ec26 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:47.390010 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91304c0d-a921-4239-80da-fdfedde5ec26 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:47.390682 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91304c0d-a921-4239-80da-fdfedde5ec26 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:47.390983 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91304c0d-a921-4239-80da-fdfedde5ec26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:47.392608 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91304c0d-a921-4239-80da-fdfedde5ec26 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:47.397876 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 312/612] 10.222.0.70 () {56 vars in 1124 bytes} [Thu Oct 27 09:24:47 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:47.398818 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91304c0d-a921-4239-80da-fdfedde5ec26 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:47.490937 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 301/613] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Oct 27 09:24:47 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 27 09:24:47.492190 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc493ac8-786e-431d-8407-2ac95dd81556 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:47.508637 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91304c0d-a921-4239-80da-fdfedde5ec26 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:47.547793 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29447569-885a-4fab-bc0f-4f99d16a2ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:47.548785 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29447569-885a-4fab-bc0f-4f99d16a2ffb None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:47.549050 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29447569-885a-4fab-bc0f-4f99d16a2ffb None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:47.549310 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29447569-885a-4fab-bc0f-4f99d16a2ffb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:47.560675 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29447569-885a-4fab-bc0f-4f99d16a2ffb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:47.562135 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29447569-885a-4fab-bc0f-4f99d16a2ffb None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:47.578983 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-29447569-885a-4fab-bc0f-4f99d16a2ffb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:47.578983 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:47.578983 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:47.578983 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:47.578983 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:47.578983 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:47.578983 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:47.578983 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:47.578983 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:47.578983 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:47.578983 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:47.578983 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:47.578983 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:24:47.578983 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:24:47.578983 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:24:47.578983 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:24:47.580031 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:47.580031 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:47.580031 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:47.580031 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:47.580031 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:47.580031 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:47.580031 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:47.580031 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:47.580031 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:47.580031 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:47.580031 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:47.580031 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:47.580031 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:47.580031 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:47.580031 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:47.580031 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:47.580031 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:24:47.581113 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:24:47.581113 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:47.581113 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:47.581658 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 313/614] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:24:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:47.582243 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29447569-885a-4fab-bc0f-4f99d16a2ffb None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:47.586928 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:47.594072 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:47.606842 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc493ac8-786e-431d-8407-2ac95dd81556 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:47.648299 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7753420e-ca9c-4f37-aa9d-63b2169b024b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:47.649091 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7753420e-ca9c-4f37-aa9d-63b2169b024b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:47.649469 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7753420e-ca9c-4f37-aa9d-63b2169b024b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:47.649867 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7753420e-ca9c-4f37-aa9d-63b2169b024b None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:47.650495 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7753420e-ca9c-4f37-aa9d-63b2169b024b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:47.650804 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7753420e-ca9c-4f37-aa9d-63b2169b024b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:47.652176 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7753420e-ca9c-4f37-aa9d-63b2169b024b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:47.657382 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 302/615] 10.222.0.70 () {56 vars in 1124 bytes} [Thu Oct 27 09:24:47 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:47.658165 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7753420e-ca9c-4f37-aa9d-63b2169b024b None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:47.688089 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29447569-885a-4fab-bc0f-4f99d16a2ffb None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:47.722763 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a270f8-15e7-4953-8949-e8bd7351ec97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:47.723408 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a270f8-15e7-4953-8949-e8bd7351ec97 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:47.723657 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a270f8-15e7-4953-8949-e8bd7351ec97 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:47.723924 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a270f8-15e7-4953-8949-e8bd7351ec97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:47.734672 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a270f8-15e7-4953-8949-e8bd7351ec97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:47.736091 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a270f8-15e7-4953-8949-e8bd7351ec97 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:47.750095 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-d2a270f8-15e7-4953-8949-e8bd7351ec97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:47.750095 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:47.750095 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:47.750095 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:47.750095 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:47.750095 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:47.750095 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:47.750095 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:47.750095 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:47.750095 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:47.750095 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:47.750095 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:47.750095 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:24:47.750095 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:24:47.750095 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:24:47.750095 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:24:47.751180 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:47.751180 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:47.751180 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:47.751180 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:47.751180 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:47.751180 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:47.751180 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:47.751180 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:47.751180 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:47.751180 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:47.751180 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:47.751180 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:47.751180 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:47.751180 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:47.751180 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:47.751180 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:47.751180 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:24:47.752063 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:24:47.752063 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:47.752063 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:47.752063 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 314/616] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:24:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:47.752063 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a270f8-15e7-4953-8949-e8bd7351ec97 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:47.764472 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7753420e-ca9c-4f37-aa9d-63b2169b024b None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:47.797734 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c5e6c7-c456-4495-9954-0fb6d8bbdf69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:47.798350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5e6c7-c456-4495-9954-0fb6d8bbdf69 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:47.798624 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5e6c7-c456-4495-9954-0fb6d8bbdf69 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:47.798890 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5e6c7-c456-4495-9954-0fb6d8bbdf69 None admin] PATH_INFO: `/v3/services/network` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:47.799456 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c5e6c7-c456-4495-9954-0fb6d8bbdf69 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:47.800844 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c5e6c7-c456-4495-9954-0fb6d8bbdf69 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:47.809139 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:47.809396 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-72c5e6c7-c456-4495-9954-0fb6d8bbdf69 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 27 09:24:47.809396 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:47.809396 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 09:24:47.809396 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 09:24:47.809396 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 09:24:47.809396 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 09:24:47.809396 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 09:24:47.809396 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:24:47.809396 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 27 09:24:47.809396 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:47.809396 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 09:24:47.809396 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:47.809396 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:47.809396 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:47.809396 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:47.809396 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:47.809396 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:47.810463 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:47.810463 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:47.810463 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:47.810463 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:47.810463 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:47.810463 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:47.810463 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 09:24:47.810463 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 09:24:47.810463 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 09:24:47.810463 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 09:24:47.810463 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:47.810463 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:47.810463 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:47.810463 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:47.810463 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:47.810463 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:47.812319 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:47.812319 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:47.812319 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:47.812319 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:47.812319 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:47.812319 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:47.812319 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:47.812319 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:47.812319 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:47.812319 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:47.812319 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 09:24:47.812319 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:24:47.812319 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 27 09:24:47.812319 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:47.812319 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 303/617] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Oct 27 09:24:47 2022] GET /identity/v3/services/network => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:47.813178 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade60ed4-a3e2-4d07-99fe-d135015efa37 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:47.813178 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade60ed4-a3e2-4d07-99fe-d135015efa37 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:47.813178 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade60ed4-a3e2-4d07-99fe-d135015efa37 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:47.813769 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 304/618] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:47.814168 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade60ed4-a3e2-4d07-99fe-d135015efa37 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:47.827683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:47.857148 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a270f8-15e7-4953-8949-e8bd7351ec97 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:47.891498 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95108c6b-ffce-468d-9da7-7f800ddfc818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:47.892099 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95108c6b-ffce-468d-9da7-7f800ddfc818 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:47.892380 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95108c6b-ffce-468d-9da7-7f800ddfc818 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:47.892685 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95108c6b-ffce-468d-9da7-7f800ddfc818 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:47.893310 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95108c6b-ffce-468d-9da7-7f800ddfc818 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:47.893613 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95108c6b-ffce-468d-9da7-7f800ddfc818 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:47.894965 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95108c6b-ffce-468d-9da7-7f800ddfc818 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:47.907929 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 315/619] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:24:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:47.908541 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95108c6b-ffce-468d-9da7-7f800ddfc818 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:47.921643 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade60ed4-a3e2-4d07-99fe-d135015efa37 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:47.955206 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3131a9-9616-437f-b0f4-9bdaf48edd84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:47.955795 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3131a9-9616-437f-b0f4-9bdaf48edd84 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:47.956065 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3131a9-9616-437f-b0f4-9bdaf48edd84 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:47.956326 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3131a9-9616-437f-b0f4-9bdaf48edd84 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:47.957011 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3131a9-9616-437f-b0f4-9bdaf48edd84 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:47.958362 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3131a9-9616-437f-b0f4-9bdaf48edd84 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:47.980389 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 305/620] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:24:47 2022] POST /identity/v3/endpoints => generated 346 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:24:47.981295 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3131a9-9616-437f-b0f4-9bdaf48edd84 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:48.014391 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95108c6b-ffce-468d-9da7-7f800ddfc818 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:48.038105 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:48.056546 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:48.060468 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c66aaf-303c-4d28-964a-c270b4802c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:48.061113 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c66aaf-303c-4d28-964a-c270b4802c67 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:48.061494 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c66aaf-303c-4d28-964a-c270b4802c67 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:48.061665 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c66aaf-303c-4d28-964a-c270b4802c67 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:48.062281 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c66aaf-303c-4d28-964a-c270b4802c67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:48.062583 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c66aaf-303c-4d28-964a-c270b4802c67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:48.065031 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c66aaf-303c-4d28-964a-c270b4802c67 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:48.079804 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 316/621] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:24:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:48.081170 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af1b9f-6825-4209-960b-329a795f6ca4 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:48.081457 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af1b9f-6825-4209-960b-329a795f6ca4 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:48.081726 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af1b9f-6825-4209-960b-329a795f6ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:48.102283 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-d1af1b9f-6825-4209-960b-329a795f6ca4 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:48.103504 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d1af1b9f-6825-4209-960b-329a795f6ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:48.105202 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3131a9-9616-437f-b0f4-9bdaf48edd84 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:48.141671 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2e2878-76e9-46c4-b641-87eb7e59307e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:48.142292 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e2878-76e9-46c4-b641-87eb7e59307e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:48.142640 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e2878-76e9-46c4-b641-87eb7e59307e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:48.142973 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e2878-76e9-46c4-b641-87eb7e59307e None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:48.143692 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2e2878-76e9-46c4-b641-87eb7e59307e None admin] RBAC: Adding query filter params (name=network) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:48.144067 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2e2878-76e9-46c4-b641-87eb7e59307e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:48.145564 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2e2878-76e9-46c4-b641-87eb7e59307e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:48.154864 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 306/622] 10.222.0.70 () {56 vars in 1112 bytes} [Thu Oct 27 09:24:47 2022] GET /identity/v3/services?name=network => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:48.154864 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2e2878-76e9-46c4-b641-87eb7e59307e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:48.184404 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 317/623] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 09:24:48.185078 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1af1b9f-6825-4209-960b-329a795f6ca4 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:48.270265 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2e2878-76e9-46c4-b641-87eb7e59307e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:48.312348 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:48.319957 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91c2553-8e1e-4210-9605-e17e848f8081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:48.320743 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c2553-8e1e-4210-9605-e17e848f8081 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:48.321141 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c2553-8e1e-4210-9605-e17e848f8081 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:48.321521 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c2553-8e1e-4210-9605-e17e848f8081 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:48.322085 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1af1b9f-6825-4209-960b-329a795f6ca4 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:48.329588 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91c2553-8e1e-4210-9605-e17e848f8081 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:48.330454 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:48.331324 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91c2553-8e1e-4210-9605-e17e848f8081 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:48.344415 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-c91c2553-8e1e-4210-9605-e17e848f8081 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 27 09:24:48.344415 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:48.344415 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:48.344415 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:48.344415 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:48.344415 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:48.344415 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:48.344415 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:48.344415 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:48.344415 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:48.344415 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:48.344415 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:48.344415 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 09:24:48.344415 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 09:24:48.344415 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 09:24:48.344415 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 09:24:48.345589 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:48.345589 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:48.345589 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 09:24:48.345589 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:24:48.345589 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 09:24:48.345589 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:24:48.345589 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:48.345589 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:48.345589 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:48.345589 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:48.345589 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:48.345589 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:48.345589 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:48.345589 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:48.345589 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:48.345589 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:48.345589 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:48.345589 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:48.346763 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:48.346763 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:48.346763 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 09:24:48.346763 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 09:24:48.346763 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 09:24:48.346763 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 09:24:48.346763 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 09:24:48.346763 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 09:24:48.346763 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 27 09:24:48.346763 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:48.349982 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 307/624] 10.222.0.70 () {56 vars in 1117 bytes} [Thu Oct 27 09:24:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:48.350519 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91c2553-8e1e-4210-9605-e17e848f8081 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:48.364962 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1b40ed-5e24-4de0-9233-14beb6732141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:48.366384 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b40ed-5e24-4de0-9233-14beb6732141 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:48.367319 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b40ed-5e24-4de0-9233-14beb6732141 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:48.368408 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b40ed-5e24-4de0-9233-14beb6732141 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:48.376447 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1b40ed-5e24-4de0-9233-14beb6732141 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:48.379067 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1b40ed-5e24-4de0-9233-14beb6732141 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:48.392878 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-1e1b40ed-5e24-4de0-9233-14beb6732141 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:24:48.392878 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:48.392878 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:48.392878 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:48.392878 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:48.392878 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:48.392878 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:48.392878 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:48.392878 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:48.392878 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:48.392878 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:48.392878 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:48.392878 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 09:24:48.392878 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 09:24:48.392878 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 09:24:48.393953 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 09:24:48.393953 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:48.393953 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:48.393953 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:48.393953 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:48.393953 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:48.393953 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:48.393953 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:48.393953 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:48.393953 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:48.393953 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:48.393953 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:48.393953 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:48.393953 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:48.393953 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:48.393953 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:48.393953 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:48.395067 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 09:24:48.395067 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 09:24:48.395067 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 09:24:48.395067 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 09:24:48.395067 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 09:24:48.395067 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 09:24:48.395067 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:24:48.395067 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:48.397120 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 318/625] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Oct 27 09:24:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:48.398024 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1b40ed-5e24-4de0-9233-14beb6732141 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:48.462627 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91c2553-8e1e-4210-9605-e17e848f8081 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:48.498211 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582da8b4-d5b4-4e46-afb6-7df06c65ced7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:48.498824 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582da8b4-d5b4-4e46-afb6-7df06c65ced7 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:48.499108 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582da8b4-d5b4-4e46-afb6-7df06c65ced7 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:48.499410 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582da8b4-d5b4-4e46-afb6-7df06c65ced7 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:48.500038 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582da8b4-d5b4-4e46-afb6-7df06c65ced7 None admin] RBAC: Adding query filter params (type=network) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:48.500343 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582da8b4-d5b4-4e46-afb6-7df06c65ced7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:48.501764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582da8b4-d5b4-4e46-afb6-7df06c65ced7 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:48.506774 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 308/626] 10.222.0.70 () {56 vars in 1112 bytes} [Thu Oct 27 09:24:48 2022] GET /identity/v3/services?type=network => generated 350 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:48.507557 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582da8b4-d5b4-4e46-afb6-7df06c65ced7 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:48.519763 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1b40ed-5e24-4de0-9233-14beb6732141 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:48.555499 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bab415c-1a71-4c59-93b6-331d5badf2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:48.556195 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab415c-1a71-4c59-93b6-331d5badf2f6 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:48.556551 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab415c-1a71-4c59-93b6-331d5badf2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:48.556933 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab415c-1a71-4c59-93b6-331d5badf2f6 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:48.557652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bab415c-1a71-4c59-93b6-331d5badf2f6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:48.557948 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bab415c-1a71-4c59-93b6-331d5badf2f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:48.559335 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bab415c-1a71-4c59-93b6-331d5badf2f6 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:48.566670 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:48.581025 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 319/627] 10.222.0.70 () {56 vars in 1149 bytes} [Thu Oct 27 09:24:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:48.581025 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bab415c-1a71-4c59-93b6-331d5badf2f6 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:48.606926 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:48.621088 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582da8b4-d5b4-4e46-afb6-7df06c65ced7 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:48.657680 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a9d329-5de3-40d5-aa59-6d91c2f99db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:48.658217 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a9d329-5de3-40d5-aa59-6d91c2f99db8 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:48.658494 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a9d329-5de3-40d5-aa59-6d91c2f99db8 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:48.658763 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a9d329-5de3-40d5-aa59-6d91c2f99db8 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:48.659448 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a9d329-5de3-40d5-aa59-6d91c2f99db8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:48.660178 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a9d329-5de3-40d5-aa59-6d91c2f99db8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:48.662010 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a9d329-5de3-40d5-aa59-6d91c2f99db8 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:48.675456 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 309/628] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Oct 27 09:24:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:48.676247 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a9d329-5de3-40d5-aa59-6d91c2f99db8 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:48.692992 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bab415c-1a71-4c59-93b6-331d5badf2f6 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:48.732689 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b3a41-8fd5-4943-8882-b62364ed715a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:48.733486 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b3a41-8fd5-4943-8882-b62364ed715a None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:48.733857 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b3a41-8fd5-4943-8882-b62364ed715a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:48.734216 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b3a41-8fd5-4943-8882-b62364ed715a None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:48.734956 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b3a41-8fd5-4943-8882-b62364ed715a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:48.736412 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b3a41-8fd5-4943-8882-b62364ed715a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:48.753170 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 320/629] 10.222.0.70 () {60 vars in 1141 bytes} [Thu Oct 27 09:24:48 2022] POST /identity/v3/services => generated 269 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:24:48.754036 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b3a41-8fd5-4943-8882-b62364ed715a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:48.794845 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a9d329-5de3-40d5-aa59-6d91c2f99db8 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:48.815734 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:48.864543 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3de7ef-0b2e-4fa7-a7b6-900cd0dfec4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:48.864543 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3de7ef-0b2e-4fa7-a7b6-900cd0dfec4d None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:48.864543 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3de7ef-0b2e-4fa7-a7b6-900cd0dfec4d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:48.864543 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3de7ef-0b2e-4fa7-a7b6-900cd0dfec4d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:48.865445 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3de7ef-0b2e-4fa7-a7b6-900cd0dfec4d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:48.865445 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3de7ef-0b2e-4fa7-a7b6-900cd0dfec4d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:48.867729 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:48.887922 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b3a41-8fd5-4943-8882-b62364ed715a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:48.892539 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 310/630] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:24:48 2022] POST /identity/v3/endpoints => generated 337 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:24:48.893649 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3de7ef-0b2e-4fa7-a7b6-900cd0dfec4d None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:48.940135 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdc12be-9751-41d5-be6c-c693825f7b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:48.940781 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc12be-9751-41d5-be6c-c693825f7b69 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:48.941064 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc12be-9751-41d5-be6c-c693825f7b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:48.941328 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc12be-9751-41d5-be6c-c693825f7b69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:49.032802 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3de7ef-0b2e-4fa7-a7b6-900cd0dfec4d None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:49.042473 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdc12be-9751-41d5-be6c-c693825f7b69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:49.043949 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdc12be-9751-41d5-be6c-c693825f7b69 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:49.073374 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c98372b-ae4f-4d4a-ad8f-e31a98ee82ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:49.073971 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c98372b-ae4f-4d4a-ad8f-e31a98ee82ae None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:49.074249 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c98372b-ae4f-4d4a-ad8f-e31a98ee82ae None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:49.074527 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c98372b-ae4f-4d4a-ad8f-e31a98ee82ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:49.075246 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c98372b-ae4f-4d4a-ad8f-e31a98ee82ae None admin] RBAC: Adding query filter params (role.id=0dfd839b7a194aee96f82f4ebc1620a7, scope.project.id=16ac205ec6f241d69d5e6b8408f09f38, user.id=eecf79ea4700496c89b1a67c614b34d0) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:49.075552 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c98372b-ae4f-4d4a-ad8f-e31a98ee82ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:49.078669 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c98372b-ae4f-4d4a-ad8f-e31a98ee82ae None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:49.087369 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 311/631] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Oct 27 09:24:48 2022] GET /identity/v3/role_assignments?user.id=eecf79ea4700496c89b1a67c614b34d0&scope.project.id=16ac205ec6f241d69d5e6b8408f09f38&role.id=0dfd839b7a194aee96f82f4ebc1620a7 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:49.163823 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:49.174184 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 321/632] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Oct 27 09:24:48 2022] GET /identity/v3/auth/tokens => generated 1271 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 27 09:24:49.180088 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c98372b-ae4f-4d4a-ad8f-e31a98ee82ae None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:49.223980 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:49.309333 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c98372b-ae4f-4d4a-ad8f-e31a98ee82ae None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:49.347399 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec08070-6811-443b-9381-2ac36a500d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:49.348037 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec08070-6811-443b-9381-2ac36a500d2c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:49.348319 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec08070-6811-443b-9381-2ac36a500d2c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:49.348610 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec08070-6811-443b-9381-2ac36a500d2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:49.359612 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec08070-6811-443b-9381-2ac36a500d2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:49.361105 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec08070-6811-443b-9381-2ac36a500d2c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:49.378581 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-6ec08070-6811-443b-9381-2ac36a500d2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:49.378581 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:49.378581 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:49.378581 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:49.378581 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:49.378581 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:49.378581 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:49.378581 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:49.378581 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:49.378581 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:49.378581 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:49.378581 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:49.378581 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:24:49.378581 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:24:49.378581 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:24:49.378581 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:24:49.379604 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:49.379604 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:49.379604 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:49.379604 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:49.379604 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:49.379604 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:49.379604 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:49.379604 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:49.379604 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:49.379604 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:49.379604 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:49.379604 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:49.379604 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:49.379604 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:49.379604 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:49.379604 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:49.379604 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:24:49.380507 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:24:49.380507 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:49.380507 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:49.384854 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 312/633] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:24:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:49.385832 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdc12be-9751-41d5-be6c-c693825f7b69 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:49.504628 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdc12be-9751-41d5-be6c-c693825f7b69 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:49.541867 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6676ee3-e76d-4535-a901-fdb3fd1fa376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:49.542822 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6676ee3-e76d-4535-a901-fdb3fd1fa376 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:49.543329 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6676ee3-e76d-4535-a901-fdb3fd1fa376 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:49.543802 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6676ee3-e76d-4535-a901-fdb3fd1fa376 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:49.544671 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6676ee3-e76d-4535-a901-fdb3fd1fa376 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:49.545175 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6676ee3-e76d-4535-a901-fdb3fd1fa376 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:49.546840 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6676ee3-e76d-4535-a901-fdb3fd1fa376 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:49.558987 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 322/634] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:24:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:49.567034 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec08070-6811-443b-9381-2ac36a500d2c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:49.653455 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:49.672781 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec08070-6811-443b-9381-2ac36a500d2c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:49.709333 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586e6400-f939-45e0-859a-cd3efc91a014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:49.710210 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e6400-f939-45e0-859a-cd3efc91a014 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:49.710671 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e6400-f939-45e0-859a-cd3efc91a014 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:49.711086 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e6400-f939-45e0-859a-cd3efc91a014 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:49.718085 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586e6400-f939-45e0-859a-cd3efc91a014 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:49.719969 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586e6400-f939-45e0-859a-cd3efc91a014 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:49.731551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-586e6400-f939-45e0-859a-cd3efc91a014 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:24:49.731551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:49.731551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:49.731551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:49.731551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:49.731551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:49.731551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:49.731551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:49.731551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:49.731551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:49.731551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:49.731551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:49.731551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 09:24:49.731551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 09:24:49.731551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 09:24:49.732676 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 09:24:49.732676 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:49.732676 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:49.732676 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:49.732676 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:49.732676 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:49.732676 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:49.732676 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:49.732676 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:49.732676 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:49.732676 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:49.732676 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:49.732676 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:49.732676 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:49.732676 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:49.732676 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:49.732676 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:49.733764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 09:24:49.733764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 09:24:49.733764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 09:24:49.733764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 09:24:49.733764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 09:24:49.733764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 09:24:49.733764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:24:49.733764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:49.735721 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 313/635] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Oct 27 09:24:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:49.742074 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6676ee3-e76d-4535-a901-fdb3fd1fa376 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:49.787480 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:49.849625 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6676ee3-e76d-4535-a901-fdb3fd1fa376 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:49.889132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709b032b-f6f6-4646-b0e2-d8bbafd161ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:49.889741 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b032b-f6f6-4646-b0e2-d8bbafd161ff None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:49.890016 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b032b-f6f6-4646-b0e2-d8bbafd161ff None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:49.890282 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b032b-f6f6-4646-b0e2-d8bbafd161ff None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:49.890962 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709b032b-f6f6-4646-b0e2-d8bbafd161ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:49.891262 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709b032b-f6f6-4646-b0e2-d8bbafd161ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:49.892706 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709b032b-f6f6-4646-b0e2-d8bbafd161ff None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:49.905396 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 323/636] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Oct 27 09:24:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:49.911396 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586e6400-f939-45e0-859a-cd3efc91a014 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:50.022345 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586e6400-f939-45e0-859a-cd3efc91a014 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:50.058764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4379442-f501-4774-a250-a0d16b565abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:50.059749 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4379442-f501-4774-a250-a0d16b565abf None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:50.060267 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4379442-f501-4774-a250-a0d16b565abf None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:50.060794 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4379442-f501-4774-a250-a0d16b565abf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:50.061769 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4379442-f501-4774-a250-a0d16b565abf None admin] RBAC: Adding query filter params (role.id=5146b19817b54113be3f81111ef9d787, scope.project.id=16ac205ec6f241d69d5e6b8408f09f38, user.id=7f389510a6a044d8a7487561b169544b) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:50.062290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4379442-f501-4774-a250-a0d16b565abf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:50.063958 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4379442-f501-4774-a250-a0d16b565abf None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:50.071145 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 314/637] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Oct 27 09:24:49 2022] GET /identity/v3/role_assignments?user.id=7f389510a6a044d8a7487561b169544b&scope.project.id=16ac205ec6f241d69d5e6b8408f09f38&role.id=5146b19817b54113be3f81111ef9d787 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:50.320418 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31582d1d-c49f-4057-adb6-3b17cbc2dd12 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:50.320816 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31582d1d-c49f-4057-adb6-3b17cbc2dd12 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:50.321187 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31582d1d-c49f-4057-adb6-3b17cbc2dd12 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:50.322883 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 324/638] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:50.328214 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b9f43-26ac-4a56-8331-4710828929f5 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:50.328642 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b9f43-26ac-4a56-8331-4710828929f5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:50.329051 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b9f43-26ac-4a56-8331-4710828929f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:50.343016 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:50.349085 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-f89b9f43-26ac-4a56-8331-4710828929f5 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:50.350827 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f89b9f43-26ac-4a56-8331-4710828929f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:50.429713 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 315/639] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:50 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 09:24:50.927428 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31582d1d-c49f-4057-adb6-3b17cbc2dd12 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:51.074242 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:51.078234 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31582d1d-c49f-4057-adb6-3b17cbc2dd12 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:51.114133 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a837c577-ee49-463e-a938-7c26c57cbc73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:51.114759 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a837c577-ee49-463e-a938-7c26c57cbc73 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:51.115028 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a837c577-ee49-463e-a938-7c26c57cbc73 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:51.115322 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a837c577-ee49-463e-a938-7c26c57cbc73 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:51.115892 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a837c577-ee49-463e-a938-7c26c57cbc73 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:51.117285 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a837c577-ee49-463e-a938-7c26c57cbc73 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:51.126123 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-a837c577-ee49-463e-a938-7c26c57cbc73 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 27 09:24:51.126123 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:51.126123 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 09:24:51.126123 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 09:24:51.126123 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 09:24:51.126123 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 09:24:51.126123 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 09:24:51.126123 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:24:51.126123 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 27 09:24:51.126123 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:51.126123 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 09:24:51.126123 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:51.126123 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:51.126123 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:51.126123 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:51.126123 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:51.126123 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:51.127120 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:51.127120 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:51.127120 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:51.127120 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:51.127120 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:51.127120 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:51.127120 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 09:24:51.127120 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 09:24:51.127120 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 09:24:51.127120 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 09:24:51.127120 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:51.127120 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:51.127120 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:51.127120 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:51.127120 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:51.127120 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:51.127948 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:51.127948 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:51.127948 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:51.127948 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:51.127948 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:51.127948 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:51.127948 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:51.127948 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:51.127948 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:51.127948 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:51.127948 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 09:24:51.127948 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:24:51.127948 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 27 09:24:51.127948 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:51.127948 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 325/640] 10.222.0.70 () {56 vars in 1114 bytes} [Thu Oct 27 09:24:50 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:51.137974 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89b9f43-26ac-4a56-8331-4710828929f5 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:51.294965 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c44568-08c9-4283-99db-2043327ea8f9 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:51.295459 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c44568-08c9-4283-99db-2043327ea8f9 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:51.295930 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c44568-08c9-4283-99db-2043327ea8f9 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:51.299998 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 326/641] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:51.306533 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e056f-b337-4ed1-8475-fdc3cba9c87b None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:51.307750 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e056f-b337-4ed1-8475-fdc3cba9c87b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:51.308815 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e056f-b337-4ed1-8475-fdc3cba9c87b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:51.313535 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89b9f43-26ac-4a56-8331-4710828929f5 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:51.345960 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-156e056f-b337-4ed1-8475-fdc3cba9c87b None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:51.348937 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-156e056f-b337-4ed1-8475-fdc3cba9c87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:51.356306 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336e7ef8-2607-4fe6-95c4-5dae18c25bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:51.357121 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e7ef8-2607-4fe6-95c4-5dae18c25bd3 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:51.357540 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e7ef8-2607-4fe6-95c4-5dae18c25bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:51.357985 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e7ef8-2607-4fe6-95c4-5dae18c25bd3 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:51.358737 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336e7ef8-2607-4fe6-95c4-5dae18c25bd3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:51.359190 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336e7ef8-2607-4fe6-95c4-5dae18c25bd3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:51.360739 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336e7ef8-2607-4fe6-95c4-5dae18c25bd3 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:51.362754 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:51.367767 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 316/642] 10.222.0.70 () {56 vars in 1114 bytes} [Thu Oct 27 09:24:51 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:51.375820 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336e7ef8-2607-4fe6-95c4-5dae18c25bd3 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:51.442314 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 327/643] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:51 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 27 09:24:51.444198 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66254b8-01d8-4cf7-b1ea-24a47f7efca9 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:51.444479 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66254b8-01d8-4cf7-b1ea-24a47f7efca9 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:51.444779 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66254b8-01d8-4cf7-b1ea-24a47f7efca9 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:51.446436 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 328/644] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:51.457690 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad836c4-3320-4edd-aa58-9c26800098b9 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:51.458078 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad836c4-3320-4edd-aa58-9c26800098b9 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:51.458450 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad836c4-3320-4edd-aa58-9c26800098b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:51.482628 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-cad836c4-3320-4edd-aa58-9c26800098b9 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:51.484488 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-cad836c4-3320-4edd-aa58-9c26800098b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:51.536615 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336e7ef8-2607-4fe6-95c4-5dae18c25bd3 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:51.568994 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 329/645] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:51 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 09:24:51.586820 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa649710-90c1-417e-9345-70aae1df837a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:51.587431 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa649710-90c1-417e-9345-70aae1df837a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:51.587710 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa649710-90c1-417e-9345-70aae1df837a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:51.587987 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa649710-90c1-417e-9345-70aae1df837a None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:51.588633 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa649710-90c1-417e-9345-70aae1df837a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:51.588939 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa649710-90c1-417e-9345-70aae1df837a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:51.590306 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa649710-90c1-417e-9345-70aae1df837a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:51.595434 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 317/646] 10.222.0.70 () {56 vars in 1114 bytes} [Thu Oct 27 09:24:51 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:51.778317 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad836c4-3320-4edd-aa58-9c26800098b9 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:51.804990 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:51.850329 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa649710-90c1-417e-9345-70aae1df837a None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:51.890243 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad836c4-3320-4edd-aa58-9c26800098b9 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:51.925530 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:51.934041 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d6d5b0-5f00-4b31-96c1-fc027b0fbdfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:51.934750 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6d5b0-5f00-4b31-96c1-fc027b0fbdfa None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:51.934971 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6d5b0-5f00-4b31-96c1-fc027b0fbdfa None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:51.935683 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6d5b0-5f00-4b31-96c1-fc027b0fbdfa None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:51.937302 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d6d5b0-5f00-4b31-96c1-fc027b0fbdfa None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:51.938880 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d6d5b0-5f00-4b31-96c1-fc027b0fbdfa None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:51.950110 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-87d6d5b0-5f00-4b31-96c1-fc027b0fbdfa None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 27 09:24:51.950110 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:51.950110 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 09:24:51.950110 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 09:24:51.950110 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 09:24:51.950110 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 09:24:51.950110 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 09:24:51.950110 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:24:51.950110 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 27 09:24:51.950110 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:51.950110 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 09:24:51.950110 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:51.950110 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:51.950110 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:51.950110 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:51.950110 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:51.951151 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:51.951151 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:51.951151 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:51.951151 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:51.951151 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:51.951151 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:51.951151 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:51.951151 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 09:24:51.951151 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 09:24:51.951151 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 09:24:51.951151 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 09:24:51.951151 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:51.951151 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:51.951151 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:51.951151 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:51.951151 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:51.951151 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:51.952132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:51.952132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:51.952132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:51.952132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:51.952132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:51.952132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:51.952132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:51.952132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:51.952132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:51.952132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:51.952132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 09:24:51.952132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:24:51.952132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 27 09:24:51.952132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:51.954983 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 330/647] 10.222.0.70 () {56 vars in 1132 bytes} [Thu Oct 27 09:24:51 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:51.963299 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d6d5b0-5f00-4b31-96c1-fc027b0fbdfa None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:51.978065 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa649710-90c1-417e-9345-70aae1df837a None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:52.019204 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190ba31-628c-49f1-b03d-c24a48e20f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:52.020145 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190ba31-628c-49f1-b03d-c24a48e20f8a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:52.020640 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190ba31-628c-49f1-b03d-c24a48e20f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:52.021080 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190ba31-628c-49f1-b03d-c24a48e20f8a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:52.022405 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f190ba31-628c-49f1-b03d-c24a48e20f8a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:52.023942 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f190ba31-628c-49f1-b03d-c24a48e20f8a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:52.033167 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-f190ba31-628c-49f1-b03d-c24a48e20f8a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 27 09:24:52.033167 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:52.033167 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 09:24:52.033167 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 09:24:52.033167 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 09:24:52.033167 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 09:24:52.033167 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 09:24:52.033167 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:24:52.033167 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 27 09:24:52.033167 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:52.033167 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 09:24:52.033167 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:52.033167 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:52.033167 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:52.033167 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:52.033167 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:52.033167 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:52.034173 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:52.034173 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:52.034173 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:52.034173 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:52.034173 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:52.034173 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:52.034173 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 09:24:52.034173 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 09:24:52.034173 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 09:24:52.034173 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 09:24:52.034173 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:52.034173 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:52.034173 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:52.034173 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:52.034173 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:52.034173 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:52.035002 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:52.035002 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:52.035002 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:52.035002 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:52.035002 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:52.035002 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:52.035002 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:52.035002 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:52.035002 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:52.035002 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:52.035002 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 09:24:52.035002 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:24:52.035002 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 27 09:24:52.035002 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:52.043268 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 318/648] 10.222.0.70 () {56 vars in 1126 bytes} [Thu Oct 27 09:24:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 09:24:52.044313 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190ba31-628c-49f1-b03d-c24a48e20f8a None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:52.072930 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:52.089330 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d6d5b0-5f00-4b31-96c1-fc027b0fbdfa None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:52.135375 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff77798c-4fab-4d21-b80e-a5033efe2f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:52.136281 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff77798c-4fab-4d21-b80e-a5033efe2f08 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:52.136662 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff77798c-4fab-4d21-b80e-a5033efe2f08 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:52.137009 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff77798c-4fab-4d21-b80e-a5033efe2f08 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:52.137705 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff77798c-4fab-4d21-b80e-a5033efe2f08 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:52.138330 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff77798c-4fab-4d21-b80e-a5033efe2f08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:52.140152 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff77798c-4fab-4d21-b80e-a5033efe2f08 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:52.148665 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 331/649] 10.222.0.70 () {56 vars in 1126 bytes} [Thu Oct 27 09:24:51 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:52.151971 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff77798c-4fab-4d21-b80e-a5033efe2f08 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:52.161387 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190ba31-628c-49f1-b03d-c24a48e20f8a None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:52.191420 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:52.202069 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20d7cf-6c05-431b-8a8d-bc06a956ebd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:52.202675 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20d7cf-6c05-431b-8a8d-bc06a956ebd5 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:52.202950 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20d7cf-6c05-431b-8a8d-bc06a956ebd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:52.203215 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20d7cf-6c05-431b-8a8d-bc06a956ebd5 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:52.203889 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa20d7cf-6c05-431b-8a8d-bc06a956ebd5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:52.204234 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa20d7cf-6c05-431b-8a8d-bc06a956ebd5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:52.205708 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa20d7cf-6c05-431b-8a8d-bc06a956ebd5 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:52.215395 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 319/650] 10.222.0.70 () {56 vars in 1122 bytes} [Thu Oct 27 09:24:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:52.216673 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20d7cf-6c05-431b-8a8d-bc06a956ebd5 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:52.288578 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff77798c-4fab-4d21-b80e-a5033efe2f08 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:52.330352 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54ee21d-e177-4993-a460-77e91ef24d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:52.331006 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ee21d-e177-4993-a460-77e91ef24d7a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:52.331290 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ee21d-e177-4993-a460-77e91ef24d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:52.331573 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ee21d-e177-4993-a460-77e91ef24d7a None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:52.332655 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54ee21d-e177-4993-a460-77e91ef24d7a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:52.332961 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54ee21d-e177-4993-a460-77e91ef24d7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:52.334363 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54ee21d-e177-4993-a460-77e91ef24d7a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:52.345557 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 332/651] 10.222.0.70 () {56 vars in 1126 bytes} [Thu Oct 27 09:24:52 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:52.346434 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54ee21d-e177-4993-a460-77e91ef24d7a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:52.368166 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:52.372547 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20d7cf-6c05-431b-8a8d-bc06a956ebd5 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:52.422345 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e56f08-5ea6-477f-bd15-390962a0e42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:52.423297 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e56f08-5ea6-477f-bd15-390962a0e42e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:52.423840 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e56f08-5ea6-477f-bd15-390962a0e42e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:52.424579 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e56f08-5ea6-477f-bd15-390962a0e42e None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:52.425989 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e56f08-5ea6-477f-bd15-390962a0e42e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:52.426496 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e56f08-5ea6-477f-bd15-390962a0e42e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:52.430445 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e56f08-5ea6-477f-bd15-390962a0e42e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:52.436253 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 320/652] 10.222.0.70 () {56 vars in 1122 bytes} [Thu Oct 27 09:24:52 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:24:52.467775 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54ee21d-e177-4993-a460-77e91ef24d7a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:52.509083 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a682af44-fd57-42ab-9eba-ad90eed24ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:52.509778 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682af44-fd57-42ab-9eba-ad90eed24ab0 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:52.510067 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682af44-fd57-42ab-9eba-ad90eed24ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:52.510344 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682af44-fd57-42ab-9eba-ad90eed24ab0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:52.511063 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a682af44-fd57-42ab-9eba-ad90eed24ab0 None admin] RBAC: Adding query filter params (interface=public, service_id=1abde16d0e0c4bef99833b3ddb1360ff, region_id=RegionOne) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:52.511376 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a682af44-fd57-42ab-9eba-ad90eed24ab0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:52.512818 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a682af44-fd57-42ab-9eba-ad90eed24ab0 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:52.519365 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 333/653] 10.222.0.70 () {56 vars in 1251 bytes} [Thu Oct 27 09:24:52 2022] GET /identity/v3/endpoints?service_id=1abde16d0e0c4bef99833b3ddb1360ff&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:52.525242 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e56f08-5ea6-477f-bd15-390962a0e42e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:52.595810 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:52.646159 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e56f08-5ea6-477f-bd15-390962a0e42e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:52.682131 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc787f8-97f1-4ca3-bf67-2f3f47e6c251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:52.682649 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc787f8-97f1-4ca3-bf67-2f3f47e6c251 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:52.682926 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc787f8-97f1-4ca3-bf67-2f3f47e6c251 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:52.683180 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc787f8-97f1-4ca3-bf67-2f3f47e6c251 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:52.683769 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc787f8-97f1-4ca3-bf67-2f3f47e6c251 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:52.685157 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc787f8-97f1-4ca3-bf67-2f3f47e6c251 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:52.692488 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 321/654] 10.222.0.70 () {56 vars in 1087 bytes} [Thu Oct 27 09:24:52 2022] GET /identity/v3/services => generated 1029 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:24:53.093776 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bb07a-abc4-4bd8-8100-007d1ac44c1d None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:53.094552 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bb07a-abc4-4bd8-8100-007d1ac44c1d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:53.094552 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bb07a-abc4-4bd8-8100-007d1ac44c1d None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:53.096215 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 334/655] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:53.101457 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90029c-773d-4ed9-9da0-076cab5b8c78 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:53.101739 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90029c-773d-4ed9-9da0-076cab5b8c78 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:53.102011 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90029c-773d-4ed9-9da0-076cab5b8c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:53.120559 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-df90029c-773d-4ed9-9da0-076cab5b8c78 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:53.121786 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-df90029c-773d-4ed9-9da0-076cab5b8c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:53.193190 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 322/656] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:53 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 09:24:53.462403 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341bb07a-abc4-4bd8-8100-007d1ac44c1d None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:53.549331 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:53.570078 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341bb07a-abc4-4bd8-8100-007d1ac44c1d None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:53.607862 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605d9803-58c7-47da-a62b-84427b2b33b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:53.608496 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d9803-58c7-47da-a62b-84427b2b33b8 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:53.609148 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d9803-58c7-47da-a62b-84427b2b33b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:53.609422 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d9803-58c7-47da-a62b-84427b2b33b8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:53.620385 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605d9803-58c7-47da-a62b-84427b2b33b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:24:53.625131 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-605d9803-58c7-47da-a62b-84427b2b33b8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 27 09:24:53.625131 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:53.625131 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:53.625131 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:53.625131 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:53.625131 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:53.625131 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:53.625131 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:53.625131 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:53.625131 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:53.625131 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:53.625131 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:53.625131 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:24:53.625131 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:24:53.625131 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:24:53.625131 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:24:53.626040 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:24:53.626040 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:24:53.626040 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:24:53.626040 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:24:53.626040 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:53.626040 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:53.626040 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:53.626040 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:53.626040 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:53.626040 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:53.626040 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:53.626040 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:53.626040 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:53.626040 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:53.626040 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:53.626040 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:53.627276 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:53.627276 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:53.627276 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:53.627276 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:53.627276 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:24:53.627276 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:24:53.627276 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:24:53.627276 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:24:53.627276 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:24:53.627276 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:24:53.627276 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 27 09:24:53.627276 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:53.627276 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 335/657] 10.222.0.70 () {56 vars in 1120 bytes} [Thu Oct 27 09:24:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:53.632372 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90029c-773d-4ed9-9da0-076cab5b8c78 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:53.688903 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:53.735797 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90029c-773d-4ed9-9da0-076cab5b8c78 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:53.771155 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faebc2a-cc41-4d1e-8c62-794fe503240c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:53.772335 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faebc2a-cc41-4d1e-8c62-794fe503240c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:53.772644 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faebc2a-cc41-4d1e-8c62-794fe503240c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:53.772914 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faebc2a-cc41-4d1e-8c62-794fe503240c None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:53.773578 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faebc2a-cc41-4d1e-8c62-794fe503240c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:53.773882 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faebc2a-cc41-4d1e-8c62-794fe503240c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:53.775245 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faebc2a-cc41-4d1e-8c62-794fe503240c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:53.783713 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 323/658] 10.222.0.70 () {56 vars in 1115 bytes} [Thu Oct 27 09:24:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:53.789343 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605d9803-58c7-47da-a62b-84427b2b33b8 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:53.891874 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605d9803-58c7-47da-a62b-84427b2b33b8 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:53.926619 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d9b548-748a-4691-b3e3-3f59f9d4f780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:53.927272 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9b548-748a-4691-b3e3-3f59f9d4f780 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:53.927554 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9b548-748a-4691-b3e3-3f59f9d4f780 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:53.927855 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9b548-748a-4691-b3e3-3f59f9d4f780 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:53.936869 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:53.941208 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d9b548-748a-4691-b3e3-3f59f9d4f780 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:53.942579 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d9b548-748a-4691-b3e3-3f59f9d4f780 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:53.957746 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-a2d9b548-748a-4691-b3e3-3f59f9d4f780 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:53.957746 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:53.957746 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:53.957746 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:53.957746 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:53.957746 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:53.957746 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:53.957746 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:53.957746 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:53.957746 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:53.957746 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:53.957746 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:53.957746 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:24:53.957746 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:24:53.957746 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:24:53.957746 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:24:53.959445 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:53.959445 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:53.959445 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:53.959445 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:53.959445 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:53.959445 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:53.959445 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:53.959445 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:53.959445 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:53.959445 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:53.959445 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:53.959445 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:53.959445 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:53.959445 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:53.959445 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:53.959445 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:53.959445 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:24:53.960585 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:24:53.960585 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:53.960585 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:53.962613 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 336/659] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:24:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:53.968291 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faebc2a-cc41-4d1e-8c62-794fe503240c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:54.077718 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faebc2a-cc41-4d1e-8c62-794fe503240c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:54.091499 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:54.113052 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd06fa6-b088-4e5f-ab30-dc90d6b51069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:54.113854 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd06fa6-b088-4e5f-ab30-dc90d6b51069 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:54.114322 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd06fa6-b088-4e5f-ab30-dc90d6b51069 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:54.114891 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd06fa6-b088-4e5f-ab30-dc90d6b51069 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:54.116113 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd06fa6-b088-4e5f-ab30-dc90d6b51069 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:54.116420 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd06fa6-b088-4e5f-ab30-dc90d6b51069 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:54.117877 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd06fa6-b088-4e5f-ab30-dc90d6b51069 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:54.121382 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b19345-3234-483f-93f5-7eb2358cd1a2 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:54.122292 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b19345-3234-483f-93f5-7eb2358cd1a2 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:54.122770 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b19345-3234-483f-93f5-7eb2358cd1a2 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:54.125635 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 337/660] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 27 09:24:54.129855 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb53e0-4168-42e0-93d9-4a5a77e5c19f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:54.130396 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb53e0-4168-42e0-93d9-4a5a77e5c19f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:54.130524 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 324/661] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:24:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:54.131555 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb53e0-4168-42e0-93d9-4a5a77e5c19f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:54.136221 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd06fa6-b088-4e5f-ab30-dc90d6b51069 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:54.151011 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f3eb53e0-4168-42e0-93d9-4a5a77e5c19f None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:54.152558 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f3eb53e0-4168-42e0-93d9-4a5a77e5c19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:54.228636 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 338/662] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:54 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 09:24:54.255623 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd06fa6-b088-4e5f-ab30-dc90d6b51069 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:54.290815 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc80a60-dcf0-4352-b076-01f3ff1e93c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:54.291443 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc80a60-dcf0-4352-b076-01f3ff1e93c9 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:54.291719 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc80a60-dcf0-4352-b076-01f3ff1e93c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:54.291990 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc80a60-dcf0-4352-b076-01f3ff1e93c9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:54.299418 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc80a60-dcf0-4352-b076-01f3ff1e93c9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:54.300906 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc80a60-dcf0-4352-b076-01f3ff1e93c9 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:54.313509 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-0dc80a60-dcf0-4352-b076-01f3ff1e93c9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 27 09:24:54.313509 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:54.313509 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:54.313509 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:54.313509 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:54.313509 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:54.313509 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:54.313509 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:54.313509 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:54.313509 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:54.313509 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:54.313509 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:54.313509 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 09:24:54.313509 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 09:24:54.313509 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 09:24:54.313509 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 09:24:54.314495 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:54.314495 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:54.314495 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 09:24:54.314495 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:24:54.314495 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 09:24:54.314495 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:24:54.314495 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:54.314495 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:54.314495 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:54.314495 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:54.314495 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:54.314495 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:54.314495 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:54.314495 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:54.314495 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:54.314495 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:54.314495 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:54.314495 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:54.316094 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:54.316094 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:54.316094 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 09:24:54.316094 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 09:24:54.316094 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 09:24:54.316094 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 09:24:54.316094 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 09:24:54.316094 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 09:24:54.316094 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 27 09:24:54.316094 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:54.316094 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 325/663] 10.222.0.70 () {56 vars in 1117 bytes} [Thu Oct 27 09:24:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 09:24:54.325201 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb53e0-4168-42e0-93d9-4a5a77e5c19f None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:54.428305 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb53e0-4168-42e0-93d9-4a5a77e5c19f None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:54.452866 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:54.464055 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac6e400-b055-4697-aac6-a1d25b7764a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:54.464706 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6e400-b055-4697-aac6-a1d25b7764a2 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:54.464986 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6e400-b055-4697-aac6-a1d25b7764a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:54.465252 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6e400-b055-4697-aac6-a1d25b7764a2 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:54.465969 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac6e400-b055-4697-aac6-a1d25b7764a2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:54.466268 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac6e400-b055-4697-aac6-a1d25b7764a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:54.467703 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac6e400-b055-4697-aac6-a1d25b7764a2 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:54.486389 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 339/664] 10.222.0.70 () {56 vars in 1149 bytes} [Thu Oct 27 09:24:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:54.492643 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc80a60-dcf0-4352-b076-01f3ff1e93c9 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:54.526448 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:54.527018 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac6e400-b055-4697-aac6-a1d25b7764a2 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:54.615437 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc80a60-dcf0-4352-b076-01f3ff1e93c9 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:54.645671 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac6e400-b055-4697-aac6-a1d25b7764a2 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:54.654539 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a44cb8d-73ab-43e8-a32e-67e2f424845e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:54.655646 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a44cb8d-73ab-43e8-a32e-67e2f424845e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:54.656126 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a44cb8d-73ab-43e8-a32e-67e2f424845e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:54.656629 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a44cb8d-73ab-43e8-a32e-67e2f424845e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:54.668893 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a44cb8d-73ab-43e8-a32e-67e2f424845e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:54.670668 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a44cb8d-73ab-43e8-a32e-67e2f424845e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:54.684801 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a8528-350d-4c43-97e7-ae3538440fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:54.685339 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a8528-350d-4c43-97e7-ae3538440fbf None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:54.685615 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a8528-350d-4c43-97e7-ae3538440fbf None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:54.685890 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a8528-350d-4c43-97e7-ae3538440fbf None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:54.686538 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3a8528-350d-4c43-97e7-ae3538440fbf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:54.687562 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-6a44cb8d-73ab-43e8-a32e-67e2f424845e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:54.687562 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:54.687562 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:54.687562 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:54.687562 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:54.687562 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:54.687562 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:54.687562 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:54.687562 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:54.687562 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:54.687562 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:54.687562 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:54.687562 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:24:54.687562 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:24:54.687562 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:24:54.687562 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:24:54.688477 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:54.688477 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:54.688477 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:54.688477 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:54.688477 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:54.688477 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:54.688477 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:54.688477 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:54.688477 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:54.688477 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:54.688477 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:54.688477 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:54.688477 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:54.688477 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:54.688477 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:54.688477 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:54.688477 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:24:54.689416 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:24:54.689416 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:54.689416 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:54.689597 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3a8528-350d-4c43-97e7-ae3538440fbf None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:54.690147 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 326/665] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:24:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:54.691235 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c55d1c-9824-4951-b70a-ab2458d06673 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:54.691516 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c55d1c-9824-4951-b70a-ab2458d06673 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:54.691783 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c55d1c-9824-4951-b70a-ab2458d06673 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:54.693795 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 327/666] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:54.695961 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c55d1c-9824-4951-b70a-ab2458d06673 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:54.712308 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 340/667] 10.222.0.70 () {60 vars in 1141 bytes} [Thu Oct 27 09:24:54 2022] POST /identity/v3/services => generated 251 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:24:54.712308 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374c006-8bfb-4d75-953c-71d1fe7f172a None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:54.712308 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374c006-8bfb-4d75-953c-71d1fe7f172a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:54.712308 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374c006-8bfb-4d75-953c-71d1fe7f172a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:54.732510 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-4374c006-8bfb-4d75-953c-71d1fe7f172a None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:54.733949 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4374c006-8bfb-4d75-953c-71d1fe7f172a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:54.780812 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:54.829826 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:54.850798 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 341/668] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:54 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 09:24:54.854717 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c55d1c-9824-4951-b70a-ab2458d06673 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:54.893438 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adc8a02-a6bd-47a8-84e3-31ff2d685a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:54.894055 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc8a02-a6bd-47a8-84e3-31ff2d685a25 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:54.894382 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc8a02-a6bd-47a8-84e3-31ff2d685a25 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:54.894660 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc8a02-a6bd-47a8-84e3-31ff2d685a25 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:54.895307 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adc8a02-a6bd-47a8-84e3-31ff2d685a25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:54.895658 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adc8a02-a6bd-47a8-84e3-31ff2d685a25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:54.897592 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adc8a02-a6bd-47a8-84e3-31ff2d685a25 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:54.908397 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04256703-e64d-4a96-a26b-6cc0840f292f None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:54.908806 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04256703-e64d-4a96-a26b-6cc0840f292f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:54.908991 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04256703-e64d-4a96-a26b-6cc0840f292f None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:54.909825 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 328/669] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:24:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:54.910309 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 342/670] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 27 09:24:54.916138 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74677902-4709-4d4d-a66d-5a1ff9bf5f35 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:54.916431 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74677902-4709-4d4d-a66d-5a1ff9bf5f35 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:54.916745 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74677902-4709-4d4d-a66d-5a1ff9bf5f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:54.917689 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04256703-e64d-4a96-a26b-6cc0840f292f None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:54.935358 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-74677902-4709-4d4d-a66d-5a1ff9bf5f35 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:54.936481 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-74677902-4709-4d4d-a66d-5a1ff9bf5f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:55.019518 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 329/671] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:54 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 09:24:55.049930 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04256703-e64d-4a96-a26b-6cc0840f292f None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:55.085627 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d89005-bb7a-4e79-af5d-f540798389b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:55.086588 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d89005-bb7a-4e79-af5d-f540798389b1 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:55.087068 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d89005-bb7a-4e79-af5d-f540798389b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:55.087540 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d89005-bb7a-4e79-af5d-f540798389b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:55.094922 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d89005-bb7a-4e79-af5d-f540798389b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:55.096595 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d89005-bb7a-4e79-af5d-f540798389b1 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:55.108590 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-62d89005-bb7a-4e79-af5d-f540798389b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:24:55.108590 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:55.108590 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:55.108590 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:55.108590 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:55.108590 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:55.108590 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:55.108590 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:55.108590 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:55.108590 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:55.108590 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:55.108590 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:55.108590 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 09:24:55.108590 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 09:24:55.108590 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 09:24:55.109494 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 09:24:55.109494 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:55.109494 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:55.109494 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:55.109494 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:55.109494 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:55.109494 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:55.109494 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:55.109494 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:55.109494 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:55.109494 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:55.109494 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:55.109494 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:55.109494 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:55.109494 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:55.109494 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:55.109494 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:55.110379 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 09:24:55.110379 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 09:24:55.110379 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 09:24:55.110379 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 09:24:55.110379 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 09:24:55.110379 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 09:24:55.110379 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:24:55.110379 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:55.116632 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 343/672] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Oct 27 09:24:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 09:24:55.118756 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74677902-4709-4d4d-a66d-5a1ff9bf5f35 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:55.133084 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d89005-bb7a-4e79-af5d-f540798389b1 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:55.164855 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:55.218260 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:55.249804 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d89005-bb7a-4e79-af5d-f540798389b1 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:55.252871 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74677902-4709-4d4d-a66d-5a1ff9bf5f35 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:55.286238 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12b3ee6-4867-43b8-9910-16c44214fc65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:55.286847 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b3ee6-4867-43b8-9910-16c44214fc65 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:55.287153 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b3ee6-4867-43b8-9910-16c44214fc65 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:55.287420 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b3ee6-4867-43b8-9910-16c44214fc65 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:55.288057 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12b3ee6-4867-43b8-9910-16c44214fc65 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:55.289454 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12b3ee6-4867-43b8-9910-16c44214fc65 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:55.303601 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e38531-7f1b-4fee-b83a-829701528177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:55.304981 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e38531-7f1b-4fee-b83a-829701528177 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:55.305462 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e38531-7f1b-4fee-b83a-829701528177 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:55.306009 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e38531-7f1b-4fee-b83a-829701528177 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:55.306936 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e38531-7f1b-4fee-b83a-829701528177 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:55.307484 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e38531-7f1b-4fee-b83a-829701528177 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:55.309303 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e38531-7f1b-4fee-b83a-829701528177 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:55.313010 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 344/673] 10.222.0.70 () {60 vars in 1141 bytes} [Thu Oct 27 09:24:55 2022] POST /identity/v3/services => generated 241 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:24:55.325053 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 330/674] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Oct 27 09:24:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:55.330048 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12b3ee6-4867-43b8-9910-16c44214fc65 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:55.331943 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e38531-7f1b-4fee-b83a-829701528177 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:55.445363 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:55.454147 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e38531-7f1b-4fee-b83a-829701528177 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:55.467347 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12b3ee6-4867-43b8-9910-16c44214fc65 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:55.495292 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22401a-ffa2-41d0-bf14-908fce77479d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:55.495878 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22401a-ffa2-41d0-bf14-908fce77479d None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:55.496154 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22401a-ffa2-41d0-bf14-908fce77479d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:55.496421 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22401a-ffa2-41d0-bf14-908fce77479d None admin] PATH_INFO: `/v3/projects/16ac205ec6f241d69d5e6b8408f09f38/users/7f389510a6a044d8a7487561b169544b/roles/77d68d4713094fd59253e4a7d20e6f33` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:55.499684 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f22401a-ffa2-41d0-bf14-908fce77479d None admin] RBAC: Authorizing `identity:create_grant(project_id=16ac205ec6f241d69d5e6b8408f09f38, user_id=7f389510a6a044d8a7487561b169544b, role_id=77d68d4713094fd59253e4a7d20e6f33)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:55.500268 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f22401a-ffa2-41d0-bf14-908fce77479d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:55.502907 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:55.512353 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b5846c-632a-4bdc-bc42-e02f9e4b454f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:55.513019 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b5846c-632a-4bdc-bc42-e02f9e4b454f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:55.513296 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b5846c-632a-4bdc-bc42-e02f9e4b454f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:55.513565 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b5846c-632a-4bdc-bc42-e02f9e4b454f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:55.514213 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b5846c-632a-4bdc-bc42-e02f9e4b454f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:55.515494 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b5846c-632a-4bdc-bc42-e02f9e4b454f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:55.517054 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 331/675] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Oct 27 09:24:55 2022] PUT /identity/v3/projects/16ac205ec6f241d69d5e6b8408f09f38/users/7f389510a6a044d8a7487561b169544b/roles/77d68d4713094fd59253e4a7d20e6f33 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:24:55.525520 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-90b5846c-632a-4bdc-bc42-e02f9e4b454f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 27 09:24:55.525520 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:55.525520 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 09:24:55.525520 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 09:24:55.525520 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 09:24:55.525520 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 09:24:55.525520 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 09:24:55.525520 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:24:55.525520 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 27 09:24:55.525520 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:55.525520 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 09:24:55.525520 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:55.525520 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:55.525520 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:55.525520 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:55.525520 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:55.526533 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:55.526533 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:55.526533 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:55.526533 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:55.526533 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:55.526533 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:55.526533 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:55.526533 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 09:24:55.526533 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 09:24:55.526533 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 09:24:55.526533 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 09:24:55.526533 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:55.526533 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:55.526533 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:55.526533 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:55.526533 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:55.526533 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:55.527859 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:55.527859 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:55.527859 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:55.527859 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:55.527859 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:55.527859 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:55.527859 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:55.527859 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:55.527859 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:55.527859 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:55.527859 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 09:24:55.527859 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:24:55.527859 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 27 09:24:55.527859 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:55.527859 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 345/676] 10.222.0.70 () {56 vars in 1132 bytes} [Thu Oct 27 09:24:55 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:55.533301 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22401a-ffa2-41d0-bf14-908fce77479d None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:55.652612 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22401a-ffa2-41d0-bf14-908fce77479d None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:55.701422 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2139c4-df5c-4d0f-a9a0-832c56f89e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:55.702108 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2139c4-df5c-4d0f-a9a0-832c56f89e46 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:55.702387 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2139c4-df5c-4d0f-a9a0-832c56f89e46 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:55.702714 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2139c4-df5c-4d0f-a9a0-832c56f89e46 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:55.703354 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2139c4-df5c-4d0f-a9a0-832c56f89e46 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:55.703656 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2139c4-df5c-4d0f-a9a0-832c56f89e46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:55.705176 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2139c4-df5c-4d0f-a9a0-832c56f89e46 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:55.711550 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 332/677] 10.222.0.70 () {56 vars in 1126 bytes} [Thu Oct 27 09:24:55 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:55.717418 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b5846c-632a-4bdc-bc42-e02f9e4b454f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:55.821237 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b5846c-632a-4bdc-bc42-e02f9e4b454f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:55.857398 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a6623-a8d9-4325-9b65-f57f33b9f284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:55.858003 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a6623-a8d9-4325-9b65-f57f33b9f284 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:55.858278 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a6623-a8d9-4325-9b65-f57f33b9f284 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:55.858545 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a6623-a8d9-4325-9b65-f57f33b9f284 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:55.859161 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7a6623-a8d9-4325-9b65-f57f33b9f284 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:55.859460 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7a6623-a8d9-4325-9b65-f57f33b9f284 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:55.860853 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7a6623-a8d9-4325-9b65-f57f33b9f284 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:55.866232 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 346/678] 10.222.0.70 () {56 vars in 1126 bytes} [Thu Oct 27 09:24:55 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:55.871725 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2139c4-df5c-4d0f-a9a0-832c56f89e46 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:55.936451 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:55.980832 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2139c4-df5c-4d0f-a9a0-832c56f89e46 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:56.015504 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81762b37-9fc8-446b-a98c-e0c72e39b7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:56.016105 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81762b37-9fc8-446b-a98c-e0c72e39b7e4 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:56.016377 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81762b37-9fc8-446b-a98c-e0c72e39b7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:56.016675 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81762b37-9fc8-446b-a98c-e0c72e39b7e4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:56.017331 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81762b37-9fc8-446b-a98c-e0c72e39b7e4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:56.018688 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81762b37-9fc8-446b-a98c-e0c72e39b7e4 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:56.042263 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 333/679] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:24:55 2022] POST /identity/v3/endpoints => generated 347 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:24:56.983524 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77069782-9969-4501-9d49-eb646ccf0374 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:56.984052 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77069782-9969-4501-9d49-eb646ccf0374 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:56.984052 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77069782-9969-4501-9d49-eb646ccf0374 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:56.985624 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 347/680] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:56.991603 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b887b-e126-46af-b511-26d77304f793 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:56.992017 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b887b-e126-46af-b511-26d77304f793 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:56.992216 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b887b-e126-46af-b511-26d77304f793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:57.010284 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-0c0b887b-e126-46af-b511-26d77304f793 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:57.011462 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0c0b887b-e126-46af-b511-26d77304f793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:57.084631 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 334/681] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 09:24:57.353913 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77069782-9969-4501-9d49-eb646ccf0374 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:57.366504 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:57.461324 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77069782-9969-4501-9d49-eb646ccf0374 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:57.501212 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71046899-a2b9-4128-b619-75a16b4fda24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:57.501935 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71046899-a2b9-4128-b619-75a16b4fda24 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:57.502214 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71046899-a2b9-4128-b619-75a16b4fda24 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:57.502479 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71046899-a2b9-4128-b619-75a16b4fda24 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:57.503064 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71046899-a2b9-4128-b619-75a16b4fda24 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:57.504454 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71046899-a2b9-4128-b619-75a16b4fda24 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:57.515078 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-71046899-a2b9-4128-b619-75a16b4fda24 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 27 09:24:57.515078 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:57.515078 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 09:24:57.515078 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 09:24:57.515078 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 09:24:57.515078 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 09:24:57.515078 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 09:24:57.515078 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:24:57.515078 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 27 09:24:57.515078 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:57.515078 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 09:24:57.515078 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:57.515078 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:57.515078 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:57.515078 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:57.515078 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:57.515078 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:57.516181 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:57.516181 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:57.516181 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:57.516181 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:57.516181 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:57.516181 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:57.516181 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 09:24:57.516181 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 09:24:57.516181 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 09:24:57.516181 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 09:24:57.516181 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:57.516181 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:57.516181 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:57.516181 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:57.516181 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:57.516181 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:57.517387 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:57.517387 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:57.517387 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:57.517387 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:57.517387 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:57.517387 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:57.517387 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:57.517387 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:57.517387 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:57.517387 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:57.517387 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 09:24:57.517387 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:24:57.517387 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 27 09:24:57.517387 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:57.517387 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 348/682] 10.222.0.70 () {56 vars in 1114 bytes} [Thu Oct 27 09:24:57 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:57.522940 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b887b-e126-46af-b511-26d77304f793 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:57.569028 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:57.624844 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b887b-e126-46af-b511-26d77304f793 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:57.636763 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0768b5-cec3-4ad3-80a5-625b440480d8 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:57.637057 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0768b5-cec3-4ad3-80a5-625b440480d8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:57.637376 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0768b5-cec3-4ad3-80a5-625b440480d8 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:57.638955 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 349/683] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 27 09:24:57.644001 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4fa6e3-0db2-4988-9550-f72087854688 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:57.644286 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4fa6e3-0db2-4988-9550-f72087854688 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:57.644576 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4fa6e3-0db2-4988-9550-f72087854688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:57.662369 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e17b1-6ca7-418e-8e8d-ac58face5d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:57.662973 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e17b1-6ca7-418e-8e8d-ac58face5d82 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:57.663252 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e17b1-6ca7-418e-8e8d-ac58face5d82 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:57.663525 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e17b1-6ca7-418e-8e8d-ac58face5d82 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:57.663832 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-4a4fa6e3-0db2-4988-9550-f72087854688 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:57.664143 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8e17b1-6ca7-418e-8e8d-ac58face5d82 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:57.664444 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8e17b1-6ca7-418e-8e8d-ac58face5d82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:57.665343 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4a4fa6e3-0db2-4988-9550-f72087854688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:57.665955 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8e17b1-6ca7-418e-8e8d-ac58face5d82 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:57.671927 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 335/684] 10.222.0.70 () {56 vars in 1114 bytes} [Thu Oct 27 09:24:57 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:57.677753 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e17b1-6ca7-418e-8e8d-ac58face5d82 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:57.755188 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 350/685] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 27 09:24:57.796416 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e17b1-6ca7-418e-8e8d-ac58face5d82 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:57.805599 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:57.832375 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b716bc-eb02-41ae-9fe9-5d50db449ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:57.833058 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b716bc-eb02-41ae-9fe9-5d50db449ea9 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:57.833338 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b716bc-eb02-41ae-9fe9-5d50db449ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:57.833610 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b716bc-eb02-41ae-9fe9-5d50db449ea9 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:57.834846 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b716bc-eb02-41ae-9fe9-5d50db449ea9 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:57.835149 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b716bc-eb02-41ae-9fe9-5d50db449ea9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:57.836548 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b716bc-eb02-41ae-9fe9-5d50db449ea9 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:57.841963 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 336/686] 10.222.0.70 () {56 vars in 1114 bytes} [Thu Oct 27 09:24:57 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:24:57.848112 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf66046-c449-4257-959a-a85854baa502 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:57.848385 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf66046-c449-4257-959a-a85854baa502 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:57.848688 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf66046-c449-4257-959a-a85854baa502 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:57.849797 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b716bc-eb02-41ae-9fe9-5d50db449ea9 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:57.850273 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 351/687] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:57.855798 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9a8c3-0188-427a-8ca0-16cceb89f8c3 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:57.856073 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9a8c3-0188-427a-8ca0-16cceb89f8c3 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:57.856388 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9a8c3-0188-427a-8ca0-16cceb89f8c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:57.870680 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:57.885924 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-72d9a8c3-0188-427a-8ca0-16cceb89f8c3 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:57.887455 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-72d9a8c3-0188-427a-8ca0-16cceb89f8c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:57.971189 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 352/688] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 09:24:57.981172 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b716bc-eb02-41ae-9fe9-5d50db449ea9 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:58.017683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b2660c-43d1-4519-8ca9-0dff5bc09255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:58.018381 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b2660c-43d1-4519-8ca9-0dff5bc09255 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:58.018637 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b2660c-43d1-4519-8ca9-0dff5bc09255 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:58.018843 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b2660c-43d1-4519-8ca9-0dff5bc09255 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:58.019532 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b2660c-43d1-4519-8ca9-0dff5bc09255 None admin] RBAC: Adding query filter params (interface=public, service_id=8d0a11933d5249bfaaaa4361c76faf23, region_id=RegionOne) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:58.019830 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b2660c-43d1-4519-8ca9-0dff5bc09255 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:58.021243 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b2660c-43d1-4519-8ca9-0dff5bc09255 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:58.029299 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 337/689] 10.222.0.70 () {56 vars in 1251 bytes} [Thu Oct 27 09:24:57 2022] GET /identity/v3/endpoints?service_id=8d0a11933d5249bfaaaa4361c76faf23&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:58.034585 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d9a8c3-0188-427a-8ca0-16cceb89f8c3 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:58.048274 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b2660c-43d1-4519-8ca9-0dff5bc09255 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:58.091209 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:58.153098 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d9a8c3-0188-427a-8ca0-16cceb89f8c3 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:58.187304 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b2660c-43d1-4519-8ca9-0dff5bc09255 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:58.189618 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc11f0c2-2dec-4e12-ba30-da59b6eedcf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:58.190254 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc11f0c2-2dec-4e12-ba30-da59b6eedcf1 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:58.190534 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc11f0c2-2dec-4e12-ba30-da59b6eedcf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:58.190807 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc11f0c2-2dec-4e12-ba30-da59b6eedcf1 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:58.191423 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc11f0c2-2dec-4e12-ba30-da59b6eedcf1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:58.192874 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc11f0c2-2dec-4e12-ba30-da59b6eedcf1 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:58.194928 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:58.201404 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 353/690] 10.222.0.70 () {56 vars in 1087 bytes} [Thu Oct 27 09:24:58 2022] GET /identity/v3/services => generated 1497 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:24:58.237113 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608fecb3-7e3c-4842-affb-ad7f2f7093d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:58.238172 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fecb3-7e3c-4842-affb-ad7f2f7093d0 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:58.238637 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fecb3-7e3c-4842-affb-ad7f2f7093d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:58.239089 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fecb3-7e3c-4842-affb-ad7f2f7093d0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:58.239961 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608fecb3-7e3c-4842-affb-ad7f2f7093d0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:58.242164 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608fecb3-7e3c-4842-affb-ad7f2f7093d0 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:58.252554 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-608fecb3-7e3c-4842-affb-ad7f2f7093d0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 27 09:24:58.252554 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:58.252554 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 09:24:58.252554 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 09:24:58.252554 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 09:24:58.252554 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 09:24:58.252554 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 09:24:58.252554 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:24:58.252554 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 27 09:24:58.252554 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:58.252554 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 09:24:58.252554 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:58.252554 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:58.252554 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:58.252554 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:58.252554 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:58.252554 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:58.253683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:58.253683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:58.253683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:58.253683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:58.253683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:58.253683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:58.253683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 09:24:58.253683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 09:24:58.253683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 09:24:58.253683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 09:24:58.253683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:58.253683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:58.253683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:58.253683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:58.253683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:58.253683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:58.254536 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:58.254536 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:58.254536 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:58.254536 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:58.254536 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:58.254536 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:58.254536 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:58.254536 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:58.254536 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:58.254536 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:58.254536 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 09:24:58.254536 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:24:58.254536 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 27 09:24:58.254536 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:58.257192 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 338/691] 10.222.0.70 () {56 vars in 1126 bytes} [Thu Oct 27 09:24:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 09:24:58.265940 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc11f0c2-2dec-4e12-ba30-da59b6eedcf1 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:58.267177 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608fecb3-7e3c-4842-affb-ad7f2f7093d0 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:58.382589 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:58.388688 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608fecb3-7e3c-4842-affb-ad7f2f7093d0 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:58.389930 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc11f0c2-2dec-4e12-ba30-da59b6eedcf1 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:58.430325 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e00e5e-b5d3-45c7-9a26-c1233b9f9ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:58.430964 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e00e5e-b5d3-45c7-9a26-c1233b9f9ef1 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:58.431247 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e00e5e-b5d3-45c7-9a26-c1233b9f9ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:58.431510 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e00e5e-b5d3-45c7-9a26-c1233b9f9ef1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:58.436883 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07d0f72-8039-46b3-9e79-dbcb3c50b082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:58.437498 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07d0f72-8039-46b3-9e79-dbcb3c50b082 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:58.437783 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07d0f72-8039-46b3-9e79-dbcb3c50b082 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:58.438028 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07d0f72-8039-46b3-9e79-dbcb3c50b082 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:58.438647 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07d0f72-8039-46b3-9e79-dbcb3c50b082 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:58.438950 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07d0f72-8039-46b3-9e79-dbcb3c50b082 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:58.440304 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07d0f72-8039-46b3-9e79-dbcb3c50b082 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:58.443175 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e00e5e-b5d3-45c7-9a26-c1233b9f9ef1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:24:58.448552 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 354/692] 10.222.0.70 () {56 vars in 1122 bytes} [Thu Oct 27 09:24:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:58.448552 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d55477-9274-42f6-8c93-b072629efdd0 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:58.448552 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d55477-9274-42f6-8c93-b072629efdd0 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:58.448833 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d55477-9274-42f6-8c93-b072629efdd0 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:58.450308 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-34e00e5e-b5d3-45c7-9a26-c1233b9f9ef1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 27 09:24:58.450308 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:58.450308 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:58.450308 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:58.450308 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:58.450308 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:58.450308 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:58.450308 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:58.450308 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:58.450308 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:58.450308 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:58.450308 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:58.450308 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:24:58.450308 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:24:58.450308 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:24:58.450308 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:24:58.451246 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:24:58.451246 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:24:58.451246 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:24:58.451246 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:24:58.451246 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:58.451246 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:58.451246 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:58.451246 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:58.451246 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:58.451246 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:58.451246 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:58.451246 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:58.451246 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:58.451246 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:58.451246 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:58.451246 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:58.452068 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:58.452068 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:58.452068 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:58.452068 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:58.452068 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:24:58.452068 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:24:58.452068 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:24:58.452068 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:24:58.452068 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:24:58.452068 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:24:58.452068 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 27 09:24:58.452068 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:58.452068 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 339/693] 10.222.0.70 () {56 vars in 1120 bytes} [Thu Oct 27 09:24:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:58.452068 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e00e5e-b5d3-45c7-9a26-c1233b9f9ef1 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:58.452852 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 355/694] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:24:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:24:58.456334 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a9eef-0b3e-4769-94e0-9ad57a8db409 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:58.456905 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a9eef-0b3e-4769-94e0-9ad57a8db409 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:58.456905 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a9eef-0b3e-4769-94e0-9ad57a8db409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:58.497477 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-187a9eef-0b3e-4769-94e0-9ad57a8db409 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:24:58.498678 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-187a9eef-0b3e-4769-94e0-9ad57a8db409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:24:58.574588 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e00e5e-b5d3-45c7-9a26-c1233b9f9ef1 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:58.579615 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:58.605152 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 356/695] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:24:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 09:24:58.607161 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187a9eef-0b3e-4769-94e0-9ad57a8db409 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:58.626339 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3121608-727d-42c5-b490-b00c1c7c1748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:58.628249 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3121608-727d-42c5-b490-b00c1c7c1748 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:58.628249 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3121608-727d-42c5-b490-b00c1c7c1748 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:58.628249 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3121608-727d-42c5-b490-b00c1c7c1748 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:58.628249 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3121608-727d-42c5-b490-b00c1c7c1748 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:58.628597 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3121608-727d-42c5-b490-b00c1c7c1748 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:58.630292 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3121608-727d-42c5-b490-b00c1c7c1748 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:58.637301 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 340/696] 10.222.0.70 () {56 vars in 1122 bytes} [Thu Oct 27 09:24:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:58.643177 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3121608-727d-42c5-b490-b00c1c7c1748 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:58.690976 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:58.766036 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187a9eef-0b3e-4769-94e0-9ad57a8db409 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:58.784663 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3121608-727d-42c5-b490-b00c1c7c1748 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:58.829325 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d178c34-2e13-44a4-8c3a-fd04884d4335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:58.832541 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d178c34-2e13-44a4-8c3a-fd04884d4335 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:58.832541 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d178c34-2e13-44a4-8c3a-fd04884d4335 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:58.832735 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33e952f-fe87-4c48-9b39-8b6fc65cfee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:58.832826 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d178c34-2e13-44a4-8c3a-fd04884d4335 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:58.832898 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e952f-fe87-4c48-9b39-8b6fc65cfee3 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:58.832967 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d178c34-2e13-44a4-8c3a-fd04884d4335 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:58.833036 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e952f-fe87-4c48-9b39-8b6fc65cfee3 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:58.833036 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e952f-fe87-4c48-9b39-8b6fc65cfee3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:58.833182 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d178c34-2e13-44a4-8c3a-fd04884d4335 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:58.833253 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33e952f-fe87-4c48-9b39-8b6fc65cfee3 None admin] RBAC: Adding query filter params (interface=public, service_id=c2231275295e48e69d2ab04ec979ff0e, region_id=RegionOne) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:58.833253 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33e952f-fe87-4c48-9b39-8b6fc65cfee3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:58.833376 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d178c34-2e13-44a4-8c3a-fd04884d4335 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:58.833840 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33e952f-fe87-4c48-9b39-8b6fc65cfee3 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:58.840063 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 341/697] 10.222.0.70 () {56 vars in 1251 bytes} [Thu Oct 27 09:24:58 2022] GET /identity/v3/endpoints?service_id=c2231275295e48e69d2ab04ec979ff0e&interface=public®ion_id=RegionOne => generated 194 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:58.846006 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33e952f-fe87-4c48-9b39-8b6fc65cfee3 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:58.846530 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 357/698] 10.222.0.70 () {56 vars in 1115 bytes} [Thu Oct 27 09:24:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:58.856546 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d178c34-2e13-44a4-8c3a-fd04884d4335 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:58.938446 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:58.958080 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:58.976085 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33e952f-fe87-4c48-9b39-8b6fc65cfee3 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:58.993271 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d178c34-2e13-44a4-8c3a-fd04884d4335 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:59.012349 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a2810b-5ec5-4757-98c1-f6668324c8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:59.012989 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a2810b-5ec5-4757-98c1-f6668324c8b4 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:59.013267 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a2810b-5ec5-4757-98c1-f6668324c8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:59.013536 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a2810b-5ec5-4757-98c1-f6668324c8b4 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:59.014137 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a2810b-5ec5-4757-98c1-f6668324c8b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:59.015502 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a2810b-5ec5-4757-98c1-f6668324c8b4 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:59.022108 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 342/699] 10.222.0.70 () {56 vars in 1087 bytes} [Thu Oct 27 09:24:58 2022] GET /identity/v3/services => generated 1497 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:24:59.022889 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a2810b-5ec5-4757-98c1-f6668324c8b4 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:59.041665 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd5830a-991d-479a-85ce-e8ac9f172817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:59.042989 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5830a-991d-479a-85ce-e8ac9f172817 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:59.043566 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5830a-991d-479a-85ce-e8ac9f172817 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:59.043913 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5830a-991d-479a-85ce-e8ac9f172817 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:59.058496 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd5830a-991d-479a-85ce-e8ac9f172817 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:59.060069 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd5830a-991d-479a-85ce-e8ac9f172817 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:59.078626 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-2bd5830a-991d-479a-85ce-e8ac9f172817 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:59.078626 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:59.078626 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:59.078626 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:59.078626 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:59.078626 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:59.078626 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:59.078626 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:59.078626 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:59.078626 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:59.078626 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:59.078626 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:59.078626 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:24:59.078626 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:24:59.078626 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:24:59.078626 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:24:59.079650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:59.079650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:59.079650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:59.079650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:59.079650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:59.079650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:59.079650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:59.079650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:59.079650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:59.079650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:59.079650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:59.079650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:59.079650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:59.079650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:59.079650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:59.079650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:59.079650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:24:59.080569 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:24:59.080569 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:24:59.080569 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:59.080569 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 358/700] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:24:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:59.088974 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd5830a-991d-479a-85ce-e8ac9f172817 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:59.153892 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a2810b-5ec5-4757-98c1-f6668324c8b4 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:59.194192 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac224ac-d04e-426c-817d-5a94779c51de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:59.194753 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac224ac-d04e-426c-817d-5a94779c51de None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:59.195032 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac224ac-d04e-426c-817d-5a94779c51de None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:59.195303 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac224ac-d04e-426c-817d-5a94779c51de None admin] PATH_INFO: `/v3/services/compute` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:59.195889 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac224ac-d04e-426c-817d-5a94779c51de None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:59.198135 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac224ac-d04e-426c-817d-5a94779c51de None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:59.209289 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-bac224ac-d04e-426c-817d-5a94779c51de None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 27 09:24:59.209289 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:59.209289 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 09:24:59.209289 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 09:24:59.209289 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 09:24:59.209289 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 09:24:59.209289 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 09:24:59.209289 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:24:59.209289 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 27 09:24:59.209289 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:59.209289 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 09:24:59.209289 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:59.209289 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:59.209289 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:59.209289 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:59.209289 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:59.209289 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:59.210372 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:59.210372 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:59.210372 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:59.210372 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:59.210372 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:59.210372 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:59.210372 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 09:24:59.210372 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 09:24:59.210372 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 09:24:59.210372 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 09:24:59.210372 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:59.210372 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:59.210372 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:59.210372 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:59.210372 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:59.210372 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:59.211220 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:59.211220 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:59.211220 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:59.211220 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:59.211220 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:59.211220 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:59.211220 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:59.211220 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:59.211220 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:59.211220 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:59.211220 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 09:24:59.211220 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:24:59.211220 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 27 09:24:59.211220 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:24:59.215778 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 343/701] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Oct 27 09:24:59 2022] GET /identity/v3/services/compute => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:24:59.216724 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac224ac-d04e-426c-817d-5a94779c51de None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:59.222742 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd5830a-991d-479a-85ce-e8ac9f172817 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:59.232611 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:59.234608 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:59.270973 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba5e38-d6dc-4167-a0a9-f753763a0981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:59.271770 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba5e38-d6dc-4167-a0a9-f753763a0981 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:59.272423 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba5e38-d6dc-4167-a0a9-f753763a0981 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:59.272423 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba5e38-d6dc-4167-a0a9-f753763a0981 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:59.273006 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ba5e38-d6dc-4167-a0a9-f753763a0981 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:59.273617 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ba5e38-d6dc-4167-a0a9-f753763a0981 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:59.275251 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ba5e38-d6dc-4167-a0a9-f753763a0981 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:59.289184 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 359/702] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:24:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:59.295036 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba5e38-d6dc-4167-a0a9-f753763a0981 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:59.350697 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac224ac-d04e-426c-817d-5a94779c51de None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:59.389678 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5abc29-711f-4180-be20-0ba85580d57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:59.390290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5abc29-711f-4180-be20-0ba85580d57f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:59.390567 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5abc29-711f-4180-be20-0ba85580d57f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:59.390837 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5abc29-711f-4180-be20-0ba85580d57f None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:59.391461 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5abc29-711f-4180-be20-0ba85580d57f None admin] RBAC: Adding query filter params (name=compute) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:59.391762 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5abc29-711f-4180-be20-0ba85580d57f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:59.393372 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5abc29-711f-4180-be20-0ba85580d57f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:59.399049 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 344/703] 10.222.0.70 () {56 vars in 1112 bytes} [Thu Oct 27 09:24:59 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:59.402149 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba5e38-d6dc-4167-a0a9-f753763a0981 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:59.404866 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5abc29-711f-4180-be20-0ba85580d57f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:59.441590 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b4633c-a1c7-4744-95ff-932509cfd879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:59.442725 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4633c-a1c7-4744-95ff-932509cfd879 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:59.443797 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4633c-a1c7-4744-95ff-932509cfd879 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:59.444346 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4633c-a1c7-4744-95ff-932509cfd879 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:59.453013 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b4633c-a1c7-4744-95ff-932509cfd879 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:59.454804 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b4633c-a1c7-4744-95ff-932509cfd879 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:59.467665 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-b0b4633c-a1c7-4744-95ff-932509cfd879 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 27 09:24:59.467665 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:24:59.467665 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:24:59.467665 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:24:59.467665 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:24:59.467665 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:24:59.467665 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:24:59.467665 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:24:59.467665 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:24:59.467665 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:24:59.467665 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:24:59.467665 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:24:59.467665 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 09:24:59.467665 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 09:24:59.467665 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 09:24:59.467665 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 09:24:59.468652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:24:59.468652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:24:59.468652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 09:24:59.468652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:24:59.468652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 09:24:59.468652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:24:59.468652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:24:59.468652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:24:59.468652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:24:59.468652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:24:59.468652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:24:59.468652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:24:59.468652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:24:59.468652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:24:59.468652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:24:59.468652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:24:59.468652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:24:59.468652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:24:59.469870 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:24:59.469870 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:24:59.469870 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 09:24:59.469870 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 09:24:59.469870 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 09:24:59.469870 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 09:24:59.469870 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 09:24:59.469870 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 09:24:59.469870 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 27 09:24:59.469870 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:24:59.476782 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 360/704] 10.222.0.70 () {56 vars in 1117 bytes} [Thu Oct 27 09:24:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 09:24:59.478405 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b4633c-a1c7-4744-95ff-932509cfd879 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:59.502952 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:59.513078 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:59.535069 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5abc29-711f-4180-be20-0ba85580d57f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:59.585194 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c330da-f16f-4a9f-a417-e087c0e65464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:59.586128 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c330da-f16f-4a9f-a417-e087c0e65464 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:59.586627 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c330da-f16f-4a9f-a417-e087c0e65464 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:59.587086 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c330da-f16f-4a9f-a417-e087c0e65464 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:59.588429 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c330da-f16f-4a9f-a417-e087c0e65464 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:59.588956 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c330da-f16f-4a9f-a417-e087c0e65464 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:59.591084 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c330da-f16f-4a9f-a417-e087c0e65464 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:59.604755 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 345/705] 10.222.0.70 () {56 vars in 1112 bytes} [Thu Oct 27 09:24:59 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:24:59.607318 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b4633c-a1c7-4744-95ff-932509cfd879 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:59.646576 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e735c2f7-13be-4568-bcc5-d93a3d7cc756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:59.647194 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e735c2f7-13be-4568-bcc5-d93a3d7cc756 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:59.647468 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e735c2f7-13be-4568-bcc5-d93a3d7cc756 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:59.647734 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e735c2f7-13be-4568-bcc5-d93a3d7cc756 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:59.648560 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e735c2f7-13be-4568-bcc5-d93a3d7cc756 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:24:59.648869 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e735c2f7-13be-4568-bcc5-d93a3d7cc756 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:59.650309 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e735c2f7-13be-4568-bcc5-d93a3d7cc756 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:24:59.670375 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 361/706] 10.222.0.70 () {56 vars in 1149 bytes} [Thu Oct 27 09:24:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:24:59.676338 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c330da-f16f-4a9f-a417-e087c0e65464 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:24:59.787530 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c330da-f16f-4a9f-a417-e087c0e65464 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:24:59.825236 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c79b97c-4d3a-47d4-813a-2e2c0ab2dc07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:24:59.825840 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c79b97c-4d3a-47d4-813a-2e2c0ab2dc07 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:24:59.826119 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c79b97c-4d3a-47d4-813a-2e2c0ab2dc07 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:24:59.826384 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c79b97c-4d3a-47d4-813a-2e2c0ab2dc07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:24:59.837370 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:24:59.900425 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c79b97c-4d3a-47d4-813a-2e2c0ab2dc07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:24:59.901934 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c79b97c-4d3a-47d4-813a-2e2c0ab2dc07 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:00.014204 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 346/707] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Oct 27 09:24:59 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 27 09:25:00.019579 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e735c2f7-13be-4568-bcc5-d93a3d7cc756 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:00.107093 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:00.125260 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e735c2f7-13be-4568-bcc5-d93a3d7cc756 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:00.161322 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b7507-93bb-46a3-a5bc-4c4a3f78523d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:00.162862 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b7507-93bb-46a3-a5bc-4c4a3f78523d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:00.163345 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b7507-93bb-46a3-a5bc-4c4a3f78523d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:00.163852 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b7507-93bb-46a3-a5bc-4c4a3f78523d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:00.174564 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b7507-93bb-46a3-a5bc-4c4a3f78523d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:00.176292 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b7507-93bb-46a3-a5bc-4c4a3f78523d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:00.191112 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-186b7507-93bb-46a3-a5bc-4c4a3f78523d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:25:00.191112 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:00.191112 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:25:00.191112 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:25:00.191112 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:25:00.191112 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:25:00.191112 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:25:00.191112 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:25:00.191112 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:25:00.191112 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:25:00.191112 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:25:00.191112 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:25:00.191112 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:25:00.191112 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:25:00.191112 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:25:00.191112 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:25:00.192079 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:25:00.192079 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:25:00.192079 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:25:00.192079 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:25:00.192079 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:25:00.192079 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:25:00.192079 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:25:00.192079 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:25:00.192079 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:25:00.192079 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:25:00.192079 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:25:00.192079 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:25:00.192079 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:25:00.192079 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:25:00.192079 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:25:00.192079 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:25:00.192079 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:25:00.193035 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:25:00.193035 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:25:00.193035 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:25:00.195108 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 362/708] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:25:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:25:00.200534 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c79b97c-4d3a-47d4-813a-2e2c0ab2dc07 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:00.244238 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:00.313560 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c79b97c-4d3a-47d4-813a-2e2c0ab2dc07 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:00.349827 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2272676e-d448-40a3-bd59-3d0a2952d507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:00.350786 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272676e-d448-40a3-bd59-3d0a2952d507 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:00.351272 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272676e-d448-40a3-bd59-3d0a2952d507 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:00.351761 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272676e-d448-40a3-bd59-3d0a2952d507 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:00.352696 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2272676e-d448-40a3-bd59-3d0a2952d507 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:25:00.353229 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2272676e-d448-40a3-bd59-3d0a2952d507 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:00.354817 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2272676e-d448-40a3-bd59-3d0a2952d507 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:00.368599 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 347/709] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:25:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:00.373724 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b7507-93bb-46a3-a5bc-4c4a3f78523d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:00.482761 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b7507-93bb-46a3-a5bc-4c4a3f78523d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:00.495565 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c5f12-2b2b-4ca0-bc55-4718199fb7ad None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:00.495873 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c5f12-2b2b-4ca0-bc55-4718199fb7ad None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:00.496115 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c5f12-2b2b-4ca0-bc55-4718199fb7ad None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:00.498040 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 348/710] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 27 09:25:00.508504 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b761bd-4eb8-4082-a99a-9d3091419f33 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:00.508815 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b761bd-4eb8-4082-a99a-9d3091419f33 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:00.509091 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b761bd-4eb8-4082-a99a-9d3091419f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:00.525868 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39215d0f-7874-4d22-85d7-8f7cd6c0d1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:00.526802 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39215d0f-7874-4d22-85d7-8f7cd6c0d1a0 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:00.527273 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39215d0f-7874-4d22-85d7-8f7cd6c0d1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:00.527772 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39215d0f-7874-4d22-85d7-8f7cd6c0d1a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:00.531041 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:00.533452 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-12b761bd-4eb8-4082-a99a-9d3091419f33 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:25:00.534721 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-12b761bd-4eb8-4082-a99a-9d3091419f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:25:00.536188 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39215d0f-7874-4d22-85d7-8f7cd6c0d1a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:00.537916 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39215d0f-7874-4d22-85d7-8f7cd6c0d1a0 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:00.550347 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-39215d0f-7874-4d22-85d7-8f7cd6c0d1a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:25:00.550347 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:00.550347 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:25:00.550347 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:25:00.550347 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:25:00.550347 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:25:00.550347 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:25:00.550347 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:25:00.550347 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:25:00.550347 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:25:00.550347 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:25:00.550347 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:25:00.550347 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 09:25:00.550347 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 09:25:00.550347 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 09:25:00.551219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 09:25:00.551219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:25:00.551219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:25:00.551219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:25:00.551219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:25:00.551219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:25:00.551219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:25:00.551219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:25:00.551219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:25:00.551219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:25:00.551219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:25:00.551219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:25:00.551219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:25:00.551219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:25:00.551219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:25:00.551219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:25:00.551219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:25:00.552125 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 09:25:00.552125 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 09:25:00.552125 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 09:25:00.552125 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 09:25:00.552125 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 09:25:00.552125 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 09:25:00.552125 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:25:00.552125 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:25:00.554422 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 363/711] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Oct 27 09:25:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 09:25:00.560132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39215d0f-7874-4d22-85d7-8f7cd6c0d1a0 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:00.615231 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 349/712] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:25:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 09:25:00.680455 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39215d0f-7874-4d22-85d7-8f7cd6c0d1a0 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:00.716021 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59faba7a-4b3b-4c54-870c-95d7bb51341f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:00.716899 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59faba7a-4b3b-4c54-870c-95d7bb51341f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:00.717317 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59faba7a-4b3b-4c54-870c-95d7bb51341f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:00.717783 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59faba7a-4b3b-4c54-870c-95d7bb51341f None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:00.718592 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59faba7a-4b3b-4c54-870c-95d7bb51341f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:25:00.719386 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59faba7a-4b3b-4c54-870c-95d7bb51341f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:00.720978 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59faba7a-4b3b-4c54-870c-95d7bb51341f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:00.733807 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 364/713] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Oct 27 09:25:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:00.740066 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b761bd-4eb8-4082-a99a-9d3091419f33 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:00.771667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:00.843859 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b761bd-4eb8-4082-a99a-9d3091419f33 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:00.882833 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b23555a-e043-47e6-a75e-9b9dcf69874c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:00.883924 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23555a-e043-47e6-a75e-9b9dcf69874c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:00.884347 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23555a-e043-47e6-a75e-9b9dcf69874c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:00.885002 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23555a-e043-47e6-a75e-9b9dcf69874c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:00.885933 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b23555a-e043-47e6-a75e-9b9dcf69874c None admin] RBAC: Adding query filter params (role.id=77d68d4713094fd59253e4a7d20e6f33, scope.project.id=16ac205ec6f241d69d5e6b8408f09f38, user.id=7f389510a6a044d8a7487561b169544b) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:25:00.886510 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b23555a-e043-47e6-a75e-9b9dcf69874c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:00.888661 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b23555a-e043-47e6-a75e-9b9dcf69874c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:00.895303 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 350/714] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Oct 27 09:25:00 2022] GET /identity/v3/role_assignments?user.id=7f389510a6a044d8a7487561b169544b&scope.project.id=16ac205ec6f241d69d5e6b8408f09f38&role.id=77d68d4713094fd59253e4a7d20e6f33 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:00.909391 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59faba7a-4b3b-4c54-870c-95d7bb51341f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:00.979101 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:01.028971 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59faba7a-4b3b-4c54-870c-95d7bb51341f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:01.067553 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49379cc-f5da-458d-8d79-34650ae23b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:01.069185 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49379cc-f5da-458d-8d79-34650ae23b31 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:01.069662 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49379cc-f5da-458d-8d79-34650ae23b31 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:01.070125 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49379cc-f5da-458d-8d79-34650ae23b31 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:01.070918 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49379cc-f5da-458d-8d79-34650ae23b31 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:01.072561 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49379cc-f5da-458d-8d79-34650ae23b31 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:01.082972 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-b49379cc-f5da-458d-8d79-34650ae23b31 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 27 09:25:01.082972 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:01.082972 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 09:25:01.082972 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 09:25:01.082972 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 09:25:01.082972 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 09:25:01.082972 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 09:25:01.082972 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:25:01.082972 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 27 09:25:01.082972 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:25:01.082972 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 09:25:01.082972 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:25:01.082972 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:01.082972 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:25:01.082972 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:25:01.082972 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:25:01.082972 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:25:01.084002 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:25:01.084002 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:25:01.084002 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:25:01.084002 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:25:01.084002 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:25:01.084002 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:25:01.084002 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 09:25:01.084002 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 09:25:01.084002 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 09:25:01.084002 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 09:25:01.084002 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:25:01.084002 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:25:01.084002 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:25:01.084002 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:25:01.084002 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:25:01.084002 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:25:01.084884 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:25:01.084884 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:25:01.084884 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:25:01.084884 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:25:01.084884 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:25:01.084884 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:25:01.084884 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:25:01.084884 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:25:01.084884 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:25:01.084884 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:25:01.084884 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 09:25:01.084884 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:25:01.084884 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 27 09:25:01.084884 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:25:01.087160 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 365/715] 10.222.0.70 () {56 vars in 1114 bytes} [Thu Oct 27 09:25:00 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:25:01.092916 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b23555a-e043-47e6-a75e-9b9dcf69874c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:01.195107 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:01.204677 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b23555a-e043-47e6-a75e-9b9dcf69874c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:01.247034 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d06ca-3121-41f5-8f98-ceca1b554d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:01.247554 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d06ca-3121-41f5-8f98-ceca1b554d9b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:01.247831 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d06ca-3121-41f5-8f98-ceca1b554d9b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:01.248091 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d06ca-3121-41f5-8f98-ceca1b554d9b None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:01.248815 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9d06ca-3121-41f5-8f98-ceca1b554d9b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:25:01.249115 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9d06ca-3121-41f5-8f98-ceca1b554d9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:01.250474 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9d06ca-3121-41f5-8f98-ceca1b554d9b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:01.255424 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 351/716] 10.222.0.70 () {56 vars in 1114 bytes} [Thu Oct 27 09:25:01 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:25:01.261197 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49379cc-f5da-458d-8d79-34650ae23b31 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:01.318986 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c4b7f-2313-469c-afda-5c6f5a6b1c3c None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:01.319343 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c4b7f-2313-469c-afda-5c6f5a6b1c3c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:01.319531 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c4b7f-2313-469c-afda-5c6f5a6b1c3c None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:01.321150 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 352/717] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:25:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 27 09:25:01.330022 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e953907-89ce-41e5-865d-16e6b9bf1947 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:01.330292 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e953907-89ce-41e5-865d-16e6b9bf1947 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:01.330557 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e953907-89ce-41e5-865d-16e6b9bf1947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:01.358149 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-9e953907-89ce-41e5-865d-16e6b9bf1947 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:25:01.359347 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9e953907-89ce-41e5-865d-16e6b9bf1947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:25:01.393909 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49379cc-f5da-458d-8d79-34650ae23b31 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:01.416176 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:01.434441 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc71ea7-f280-4f06-b1a5-d018635d8771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:01.435048 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc71ea7-f280-4f06-b1a5-d018635d8771 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:01.435329 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc71ea7-f280-4f06-b1a5-d018635d8771 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:01.435600 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc71ea7-f280-4f06-b1a5-d018635d8771 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:01.436325 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc71ea7-f280-4f06-b1a5-d018635d8771 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:25:01.436670 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc71ea7-f280-4f06-b1a5-d018635d8771 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:01.438164 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc71ea7-f280-4f06-b1a5-d018635d8771 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:01.444687 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 366/718] 10.222.0.70 () {56 vars in 1114 bytes} [Thu Oct 27 09:25:01 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:01.450373 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc71ea7-f280-4f06-b1a5-d018635d8771 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:01.451687 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 353/719] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:25:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 09:25:01.555012 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc71ea7-f280-4f06-b1a5-d018635d8771 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:01.590010 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f784b859-fff2-4158-88b1-9ba2d1783a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:01.590614 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784b859-fff2-4158-88b1-9ba2d1783a1b None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:01.590893 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784b859-fff2-4158-88b1-9ba2d1783a1b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:01.591165 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784b859-fff2-4158-88b1-9ba2d1783a1b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:01.591828 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f784b859-fff2-4158-88b1-9ba2d1783a1b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:01.593202 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f784b859-fff2-4158-88b1-9ba2d1783a1b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:01.617669 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 367/720] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:25:01 2022] POST /identity/v3/endpoints => generated 346 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:25:01.723849 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e953907-89ce-41e5-865d-16e6b9bf1947 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:01.822153 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:01.835677 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e953907-89ce-41e5-865d-16e6b9bf1947 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:01.885536 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db0501-08e7-4f12-94d6-07e1e8aee00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:01.886182 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db0501-08e7-4f12-94d6-07e1e8aee00e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:01.886458 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db0501-08e7-4f12-94d6-07e1e8aee00e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:01.886727 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db0501-08e7-4f12-94d6-07e1e8aee00e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:01.887308 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26db0501-08e7-4f12-94d6-07e1e8aee00e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:01.888714 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26db0501-08e7-4f12-94d6-07e1e8aee00e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:01.898625 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-26db0501-08e7-4f12-94d6-07e1e8aee00e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 27 09:25:01.898625 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:01.898625 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 09:25:01.898625 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 09:25:01.898625 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 09:25:01.898625 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 09:25:01.898625 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 09:25:01.898625 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:25:01.898625 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 27 09:25:01.898625 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:25:01.898625 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 09:25:01.898625 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:25:01.898625 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:01.898625 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:25:01.898625 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:25:01.898625 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:25:01.898625 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:25:01.899644 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:25:01.899644 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:25:01.899644 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:25:01.899644 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:25:01.899644 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:25:01.899644 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:25:01.899644 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 09:25:01.899644 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 09:25:01.899644 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 09:25:01.899644 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 09:25:01.899644 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:25:01.899644 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:25:01.899644 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:25:01.899644 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:25:01.899644 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:25:01.899644 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:25:01.901755 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:25:01.901755 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:25:01.901755 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:25:01.901755 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:25:01.901755 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:25:01.901755 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:25:01.901755 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:25:01.901755 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:25:01.901755 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:25:01.901755 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:25:01.901755 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 09:25:01.901755 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:25:01.901755 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 27 09:25:01.901755 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:25:01.901755 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 354/721] 10.222.0.70 () {56 vars in 1126 bytes} [Thu Oct 27 09:25:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:25:01.905976 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f784b859-fff2-4158-88b1-9ba2d1783a1b None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:01.951942 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:01.981617 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c56ac-7532-45f8-9827-36bc5977ddb0 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:01.981937 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c56ac-7532-45f8-9827-36bc5977ddb0 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:01.982157 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c56ac-7532-45f8-9827-36bc5977ddb0 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:01.983722 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 355/722] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:25:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:25:01.988791 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503cec1-3bd3-4bfc-8965-7898384d87ba None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:01.989069 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503cec1-3bd3-4bfc-8965-7898384d87ba None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:01.989341 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503cec1-3bd3-4bfc-8965-7898384d87ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:02.007965 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a503cec1-3bd3-4bfc-8965-7898384d87ba None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:25:02.009210 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a503cec1-3bd3-4bfc-8965-7898384d87ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:25:02.023714 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f784b859-fff2-4158-88b1-9ba2d1783a1b None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:02.065717 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca11eb1d-b23a-40e9-b847-85f72294bb41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:02.066468 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca11eb1d-b23a-40e9-b847-85f72294bb41 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:02.066763 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca11eb1d-b23a-40e9-b847-85f72294bb41 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:02.067034 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca11eb1d-b23a-40e9-b847-85f72294bb41 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:02.068634 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca11eb1d-b23a-40e9-b847-85f72294bb41 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:25:02.068978 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca11eb1d-b23a-40e9-b847-85f72294bb41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:02.070355 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca11eb1d-b23a-40e9-b847-85f72294bb41 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:02.075580 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 368/723] 10.222.0.70 () {56 vars in 1122 bytes} [Thu Oct 27 09:25:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:02.080059 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 356/724] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:25:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 27 09:25:02.081530 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca11eb1d-b23a-40e9-b847-85f72294bb41 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:02.181728 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca11eb1d-b23a-40e9-b847-85f72294bb41 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:02.184962 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:02.218607 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346cfebe-fe6a-4f7a-a685-3bccb2f224f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:02.219535 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346cfebe-fe6a-4f7a-a685-3bccb2f224f7 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:02.219814 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346cfebe-fe6a-4f7a-a685-3bccb2f224f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:02.220084 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346cfebe-fe6a-4f7a-a685-3bccb2f224f7 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:02.220732 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346cfebe-fe6a-4f7a-a685-3bccb2f224f7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:25:02.221033 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346cfebe-fe6a-4f7a-a685-3bccb2f224f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:02.222470 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346cfebe-fe6a-4f7a-a685-3bccb2f224f7 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:02.227719 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 369/725] 10.222.0.70 () {56 vars in 1122 bytes} [Thu Oct 27 09:25:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:25:02.233552 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a503cec1-3bd3-4bfc-8965-7898384d87ba None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:02.319138 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:02.343435 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a503cec1-3bd3-4bfc-8965-7898384d87ba None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:02.357324 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346cfebe-fe6a-4f7a-a685-3bccb2f224f7 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:02.389802 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c05cef-50e6-46b0-a1d9-9693c4c8d13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:02.390688 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c05cef-50e6-46b0-a1d9-9693c4c8d13b None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:02.390963 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c05cef-50e6-46b0-a1d9-9693c4c8d13b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:02.391231 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c05cef-50e6-46b0-a1d9-9693c4c8d13b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:02.392061 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c05cef-50e6-46b0-a1d9-9693c4c8d13b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:02.394158 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c05cef-50e6-46b0-a1d9-9693c4c8d13b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:02.419439 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 357/726] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:25:02 2022] POST /identity/v3/endpoints => generated 349 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:25:02.480597 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346cfebe-fe6a-4f7a-a685-3bccb2f224f7 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:02.527173 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9931b54d-ffc8-4b58-88f0-2b78d0daafaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:02.527792 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9931b54d-ffc8-4b58-88f0-2b78d0daafaf None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:02.528136 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9931b54d-ffc8-4b58-88f0-2b78d0daafaf None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:02.528410 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9931b54d-ffc8-4b58-88f0-2b78d0daafaf None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:02.529095 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9931b54d-ffc8-4b58-88f0-2b78d0daafaf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:02.530549 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9931b54d-ffc8-4b58-88f0-2b78d0daafaf None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:02.549724 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 370/727] 10.222.0.70 () {60 vars in 1141 bytes} [Thu Oct 27 09:25:02 2022] POST /identity/v3/services => generated 242 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:25:03.206029 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa606a49-7ecc-41d1-bad7-e90cffdacdf0 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:03.206366 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa606a49-7ecc-41d1-bad7-e90cffdacdf0 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:03.206633 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa606a49-7ecc-41d1-bad7-e90cffdacdf0 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:03.208272 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 358/728] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:25:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:25:03.214223 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e08da7-95fd-4ddc-bf89-3c6856ca77b5 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:03.214517 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e08da7-95fd-4ddc-bf89-3c6856ca77b5 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:03.214797 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e08da7-95fd-4ddc-bf89-3c6856ca77b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:03.234650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-92e08da7-95fd-4ddc-bf89-3c6856ca77b5 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:25:03.235844 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-92e08da7-95fd-4ddc-bf89-3c6856ca77b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:25:03.254164 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:03.316581 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 371/729] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:25:03 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 09:25:03.588814 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa606a49-7ecc-41d1-bad7-e90cffdacdf0 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:03.695418 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa606a49-7ecc-41d1-bad7-e90cffdacdf0 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:03.730302 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a387e0b6-3436-4d7a-9b95-01f64c523cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:03.730931 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387e0b6-3436-4d7a-9b95-01f64c523cbe None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:03.731205 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387e0b6-3436-4d7a-9b95-01f64c523cbe None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:03.731471 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387e0b6-3436-4d7a-9b95-01f64c523cbe None admin] PATH_INFO: `/v3/services/image` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:03.732050 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a387e0b6-3436-4d7a-9b95-01f64c523cbe None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:03.733514 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a387e0b6-3436-4d7a-9b95-01f64c523cbe None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:03.735362 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:03.742998 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-a387e0b6-3436-4d7a-9b95-01f64c523cbe None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 27 09:25:03.742998 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:03.742998 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 09:25:03.742998 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 09:25:03.742998 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 09:25:03.742998 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 09:25:03.742998 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 09:25:03.742998 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:25:03.742998 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 27 09:25:03.742998 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:25:03.742998 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 09:25:03.742998 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:25:03.742998 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:03.742998 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:25:03.742998 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:25:03.742998 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:25:03.742998 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:25:03.744286 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:25:03.744286 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:25:03.744286 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:25:03.744286 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:25:03.744286 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:25:03.744286 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:25:03.744286 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 09:25:03.744286 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 09:25:03.744286 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 09:25:03.744286 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 09:25:03.744286 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:25:03.744286 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:25:03.744286 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:25:03.744286 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:25:03.744286 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:25:03.744286 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:25:03.745621 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:25:03.745621 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:25:03.745621 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:25:03.745621 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:25:03.745621 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:25:03.745621 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:25:03.745621 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:25:03.745621 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:25:03.745621 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:25:03.745621 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:25:03.745621 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 09:25:03.745621 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:25:03.745621 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 27 09:25:03.745621 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:25:03.745621 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 359/730] 10.222.0.70 () {56 vars in 1105 bytes} [Thu Oct 27 09:25:03 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:25:03.753974 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e08da7-95fd-4ddc-bf89-3c6856ca77b5 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:03.863212 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e08da7-95fd-4ddc-bf89-3c6856ca77b5 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:03.900158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72da8be-cd64-4257-bf7e-8fa3d3e291a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:03.901137 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72da8be-cd64-4257-bf7e-8fa3d3e291a8 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:03.901646 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72da8be-cd64-4257-bf7e-8fa3d3e291a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:03.902117 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72da8be-cd64-4257-bf7e-8fa3d3e291a8 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:03.902937 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72da8be-cd64-4257-bf7e-8fa3d3e291a8 None admin] RBAC: Adding query filter params (name=image) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:25:03.903435 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72da8be-cd64-4257-bf7e-8fa3d3e291a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:03.905020 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72da8be-cd64-4257-bf7e-8fa3d3e291a8 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:03.914759 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 372/731] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:25:03 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:03.917125 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a387e0b6-3436-4d7a-9b95-01f64c523cbe None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:03.932086 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619cb0e-a523-41ea-a0cd-f7f0d6c2a105 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:03.932691 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619cb0e-a523-41ea-a0cd-f7f0d6c2a105 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:03.933187 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619cb0e-a523-41ea-a0cd-f7f0d6c2a105 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:03.934983 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 373/732] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:25:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 27 09:25:03.940632 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ada768-02cc-4636-9604-47f2a3719737 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:03.941173 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ada768-02cc-4636-9604-47f2a3719737 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:03.941661 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ada768-02cc-4636-9604-47f2a3719737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:03.960467 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-15ada768-02cc-4636-9604-47f2a3719737 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:25:03.962006 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-15ada768-02cc-4636-9604-47f2a3719737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:25:03.972340 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:04.038614 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a387e0b6-3436-4d7a-9b95-01f64c523cbe None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:04.051977 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 374/733] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:25:03 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 27 09:25:04.080922 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a20674-28ab-4e16-a751-74b356a4f9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:04.081678 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a20674-28ab-4e16-a751-74b356a4f9d2 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:04.082073 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a20674-28ab-4e16-a751-74b356a4f9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:04.082478 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a20674-28ab-4e16-a751-74b356a4f9d2 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:04.083200 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a20674-28ab-4e16-a751-74b356a4f9d2 None admin] RBAC: Adding query filter params (type=image) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:25:04.083630 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a20674-28ab-4e16-a751-74b356a4f9d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:04.085127 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a20674-28ab-4e16-a751-74b356a4f9d2 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:04.093804 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 360/734] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:25:03 2022] GET /identity/v3/services?type=image => generated 122 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:04.327290 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ada768-02cc-4636-9604-47f2a3719737 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:04.435632 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ada768-02cc-4636-9604-47f2a3719737 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:04.471474 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e0ceb-d46c-4365-8079-ca7a4938f2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:04.472453 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e0ceb-d46c-4365-8079-ca7a4938f2ad None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:04.472955 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e0ceb-d46c-4365-8079-ca7a4938f2ad None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:04.473435 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e0ceb-d46c-4365-8079-ca7a4938f2ad None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:04.474800 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153e0ceb-d46c-4365-8079-ca7a4938f2ad None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:04.476416 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153e0ceb-d46c-4365-8079-ca7a4938f2ad None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:04.480492 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: WARNING keystone.server.flask.common [None req-153e0ceb-d46c-4365-8079-ca7a4938f2ad None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 27 09:25:04.495694 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:04.502480 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.sql.core [None req-153e0ceb-d46c-4365-8079-ca7a4938f2ad None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 27 09:25:04.502480 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 27 09:25:04.502480 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [parameters: {'id': 'd4890fd1dc4c4e5bacedf354b96fc166', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 27 09:25:04.502480 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81436) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 27 09:25:04.511862 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-153e0ceb-d46c-4365-8079-ca7a4938f2ad None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d4890fd1dc4c4e5bacedf354b96fc166.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d4890fd1dc4c4e5bacedf354b96fc166. Oct 27 09:25:04.511862 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:04.511862 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 27 09:25:04.511862 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 27 09:25:04.511862 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 27 09:25:04.511862 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 27 09:25:04.511862 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 27 09:25:04.511862 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application result = self._query(query) Oct 27 09:25:04.511862 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 27 09:25:04.511862 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application conn.query(q) Oct 27 09:25:04.511862 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 27 09:25:04.511862 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 27 09:25:04.511862 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 27 09:25:04.512707 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application result.read() Oct 27 09:25:04.512707 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 27 09:25:04.512707 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 27 09:25:04.512707 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 27 09:25:04.512707 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 27 09:25:04.512707 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 27 09:25:04.512707 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 27 09:25:04.512707 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 27 09:25:04.512707 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 27 09:25:04.512707 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 27 09:25:04.512707 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:25:04.512707 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 27 09:25:04.512707 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:25:04.512707 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:04.512707 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 27 09:25:04.512707 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 27 09:25:04.512707 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 27 09:25:04.512707 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 27 09:25:04.513700 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 27 09:25:04.513700 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application next(self.gen) Oct 27 09:25:04.513700 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 27 09:25:04.513700 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application yield resource Oct 27 09:25:04.513700 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 27 09:25:04.513700 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application next(self.gen) Oct 27 09:25:04.513700 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 27 09:25:04.513700 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self.session.rollback() Oct 27 09:25:04.513700 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 27 09:25:04.513700 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self.force_reraise() Oct 27 09:25:04.513700 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 27 09:25:04.513700 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise self.value Oct 27 09:25:04.513700 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 27 09:25:04.513700 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 27 09:25:04.513700 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 27 09:25:04.513700 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application session.commit() Oct 27 09:25:04.513700 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 27 09:25:04.514586 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 27 09:25:04.514586 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 27 09:25:04.514586 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._prepare_impl() Oct 27 09:25:04.514586 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 27 09:25:04.514586 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self.session.flush() Oct 27 09:25:04.514586 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 27 09:25:04.514586 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._flush(objects) Oct 27 09:25:04.514586 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 27 09:25:04.514586 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 27 09:25:04.514586 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 27 09:25:04.514586 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application compat.raise_( Oct 27 09:25:04.514586 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 27 09:25:04.514586 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception Oct 27 09:25:04.514586 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 27 09:25:04.514586 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application flush_context.execute() Oct 27 09:25:04.514586 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 27 09:25:04.514586 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rec.execute(self) Oct 27 09:25:04.515495 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 27 09:25:04.515495 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 27 09:25:04.515495 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 27 09:25:04.515495 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 27 09:25:04.515495 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 27 09:25:04.515495 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 27 09:25:04.515495 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 27 09:25:04.515495 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 27 09:25:04.515495 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 27 09:25:04.515495 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 27 09:25:04.515495 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 27 09:25:04.515495 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 27 09:25:04.515495 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 27 09:25:04.515495 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 27 09:25:04.515495 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 27 09:25:04.516288 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 27 09:25:04.516288 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 27 09:25:04.516288 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception Oct 27 09:25:04.516288 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 27 09:25:04.516288 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 27 09:25:04.516288 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 27 09:25:04.516288 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 27 09:25:04.516288 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 27 09:25:04.516288 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application result = self._query(query) Oct 27 09:25:04.516288 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 27 09:25:04.516288 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application conn.query(q) Oct 27 09:25:04.516288 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 27 09:25:04.516288 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 27 09:25:04.516288 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 27 09:25:04.516288 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application result.read() Oct 27 09:25:04.516288 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 27 09:25:04.516288 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 27 09:25:04.518975 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 27 09:25:04.518975 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 27 09:25:04.518975 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 27 09:25:04.518975 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 27 09:25:04.518975 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 27 09:25:04.518975 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 27 09:25:04.518975 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 27 09:25:04.518975 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 27 09:25:04.518975 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [parameters: {'id': 'd4890fd1dc4c4e5bacedf354b96fc166', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 27 09:25:04.518975 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 27 09:25:04.518975 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:25:04.518975 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 09:25:04.518975 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:25:04.518975 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:04.518975 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 27 09:25:04.520900 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 27 09:25:04.520900 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 27 09:25:04.520900 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 27 09:25:04.520900 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 27 09:25:04.520900 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:25:04.520900 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 09:25:04.520900 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:25:04.520900 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:04.520900 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:25:04.520900 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:25:04.520900 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:25:04.520900 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:25:04.520900 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:25:04.520900 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:25:04.520900 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:25:04.520900 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:25:04.520900 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:25:04.523135 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:25:04.523135 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 27 09:25:04.523135 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 27 09:25:04.523135 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:25:04.523135 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:25:04.523135 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 27 09:25:04.523135 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 27 09:25:04.523135 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d4890fd1dc4c4e5bacedf354b96fc166. Oct 27 09:25:04.523135 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:25:04.526057 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 375/735] 10.222.0.70 () {60 vars in 1140 bytes} [Thu Oct 27 09:25:04 2022] POST /identity/v3/projects => generated 262 bytes in 199 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:25:04.532574 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a20674-28ab-4e16-a751-74b356a4f9d2 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:04.599853 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:04.644677 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a20674-28ab-4e16-a751-74b356a4f9d2 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:04.680725 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f530e0-1222-4c5c-b7ae-a191a33811a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:04.681350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f530e0-1222-4c5c-b7ae-a191a33811a6 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:04.681623 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f530e0-1222-4c5c-b7ae-a191a33811a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:04.681889 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f530e0-1222-4c5c-b7ae-a191a33811a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:04.689041 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f530e0-1222-4c5c-b7ae-a191a33811a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:04.690537 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f530e0-1222-4c5c-b7ae-a191a33811a6 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:04.703052 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-81f530e0-1222-4c5c-b7ae-a191a33811a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:25:04.703052 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:04.703052 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:25:04.703052 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:25:04.703052 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:25:04.703052 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:25:04.703052 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:25:04.703052 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:25:04.703052 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:25:04.703052 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:25:04.703052 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:25:04.703052 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:25:04.703052 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 09:25:04.703052 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 09:25:04.703052 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 09:25:04.704014 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 09:25:04.704014 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:25:04.704014 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:25:04.704014 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:25:04.704014 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:25:04.704014 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:25:04.704014 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:25:04.704014 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:25:04.704014 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:25:04.704014 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:25:04.704014 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:25:04.704014 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:25:04.704014 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:25:04.704014 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:25:04.704014 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:25:04.704014 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:25:04.704014 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:25:04.704945 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 09:25:04.704945 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 09:25:04.704945 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 09:25:04.704945 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 09:25:04.704945 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 09:25:04.704945 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 09:25:04.704945 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:25:04.704945 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:25:04.704945 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 361/736] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Oct 27 09:25:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:25:04.711564 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e0ceb-d46c-4365-8079-ca7a4938f2ad None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:04.737730 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b325c44-979b-46c2-8b39-3289852ac206 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:04.738079 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b325c44-979b-46c2-8b39-3289852ac206 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:04.738289 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b325c44-979b-46c2-8b39-3289852ac206 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:04.739936 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 362/737] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:25:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:25:04.745348 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d97590-fcc9-42b5-a2d6-8b4aba7544f2 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:04.745635 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d97590-fcc9-42b5-a2d6-8b4aba7544f2 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:04.745908 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d97590-fcc9-42b5-a2d6-8b4aba7544f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:04.764115 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-c3d97590-fcc9-42b5-a2d6-8b4aba7544f2 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:25:04.765639 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c3d97590-fcc9-42b5-a2d6-8b4aba7544f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:25:04.841997 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e0ceb-d46c-4365-8079-ca7a4938f2ad None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:04.844796 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 363/738] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:25:04 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 27 09:25:04.846017 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb0f255-9149-4e93-9cf5-c207ab826c48 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:04.846316 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb0f255-9149-4e93-9cf5-c207ab826c48 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:04.846583 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb0f255-9149-4e93-9cf5-c207ab826c48 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:04.847961 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 364/739] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:25:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:25:04.857836 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c617794-c99a-4959-a0d8-ce34358ab923 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:04.858073 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c617794-c99a-4959-a0d8-ce34358ab923 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:04.858323 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c617794-c99a-4959-a0d8-ce34358ab923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:04.879263 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-7c617794-c99a-4959-a0d8-ce34358ab923 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:25:04.879590 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518a291d-28ff-4ee1-94e1-ff18ced102ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:04.880393 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a291d-28ff-4ee1-94e1-ff18ced102ea None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:04.880510 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7c617794-c99a-4959-a0d8-ce34358ab923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:25:04.880956 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a291d-28ff-4ee1-94e1-ff18ced102ea None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:04.881368 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a291d-28ff-4ee1-94e1-ff18ced102ea None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:04.882166 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518a291d-28ff-4ee1-94e1-ff18ced102ea None admin] RBAC: Adding query filter params (name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:25:04.882877 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518a291d-28ff-4ee1-94e1-ff18ced102ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:04.885625 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518a291d-28ff-4ee1-94e1-ff18ced102ea None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:04.916562 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 376/740] 10.222.0.70 () {56 vars in 1112 bytes} [Thu Oct 27 09:25:04 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:04.945737 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:04.969984 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 365/741] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:25:04 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 27 09:25:05.134116 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518a291d-28ff-4ee1-94e1-ff18ced102ea None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:05.199378 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:05.243191 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518a291d-28ff-4ee1-94e1-ff18ced102ea None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:05.244161 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c617794-c99a-4959-a0d8-ce34358ab923 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:05.280800 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800358ae-6e9d-4ec3-963a-49f1a217c054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:05.281438 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800358ae-6e9d-4ec3-963a-49f1a217c054 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:05.281771 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800358ae-6e9d-4ec3-963a-49f1a217c054 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:05.282049 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800358ae-6e9d-4ec3-963a-49f1a217c054 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:05.282625 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800358ae-6e9d-4ec3-963a-49f1a217c054 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:05.283991 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800358ae-6e9d-4ec3-963a-49f1a217c054 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:05.293376 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-800358ae-6e9d-4ec3-963a-49f1a217c054 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 27 09:25:05.293376 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:05.293376 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 09:25:05.293376 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 09:25:05.293376 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 09:25:05.293376 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 09:25:05.293376 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 09:25:05.293376 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:25:05.293376 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 27 09:25:05.293376 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:25:05.293376 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 09:25:05.293376 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:25:05.293376 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:05.293376 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:25:05.293376 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:25:05.293376 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:25:05.293376 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:25:05.294412 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:25:05.294412 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:25:05.294412 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:25:05.294412 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:25:05.294412 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:25:05.294412 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:25:05.294412 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 09:25:05.294412 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 09:25:05.294412 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 09:25:05.294412 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 09:25:05.294412 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:25:05.294412 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:25:05.294412 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:25:05.294412 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:25:05.294412 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:25:05.294412 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:25:05.295248 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:25:05.295248 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:25:05.295248 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:25:05.295248 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:25:05.295248 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:25:05.295248 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:25:05.295248 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:25:05.295248 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:25:05.295248 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:25:05.295248 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:25:05.295248 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 09:25:05.295248 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:25:05.295248 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 27 09:25:05.295248 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:25:05.295248 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 377/742] 10.222.0.70 () {56 vars in 1126 bytes} [Thu Oct 27 09:25:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:25:05.300560 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800358ae-6e9d-4ec3-963a-49f1a217c054 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:05.374301 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c617794-c99a-4959-a0d8-ce34358ab923 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:05.414727 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800358ae-6e9d-4ec3-963a-49f1a217c054 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:05.420052 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25936b1c-6249-4b5d-ae2e-65cc018063d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:05.420715 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25936b1c-6249-4b5d-ae2e-65cc018063d3 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:05.420998 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25936b1c-6249-4b5d-ae2e-65cc018063d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:05.421268 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25936b1c-6249-4b5d-ae2e-65cc018063d3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:05.421844 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25936b1c-6249-4b5d-ae2e-65cc018063d3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:05.423751 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25936b1c-6249-4b5d-ae2e-65cc018063d3 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:05.436045 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-25936b1c-6249-4b5d-ae2e-65cc018063d3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 27 09:25:05.436045 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:05.436045 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 09:25:05.436045 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 09:25:05.436045 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 09:25:05.436045 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 09:25:05.436045 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 09:25:05.436045 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:25:05.436045 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 27 09:25:05.436045 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:25:05.436045 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 09:25:05.436045 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:25:05.436045 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:05.436045 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:25:05.436045 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:25:05.436045 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:25:05.436045 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:25:05.437107 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:25:05.437107 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:25:05.437107 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:25:05.437107 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:25:05.437107 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:25:05.437107 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:25:05.437107 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 09:25:05.437107 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 09:25:05.437107 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 09:25:05.437107 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 09:25:05.437107 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:25:05.437107 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:25:05.437107 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:25:05.437107 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:25:05.437107 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:25:05.437107 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:25:05.437935 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:25:05.437935 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:25:05.437935 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:25:05.437935 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:25:05.437935 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:25:05.437935 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:25:05.437935 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:25:05.437935 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:25:05.437935 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:25:05.437935 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:25:05.437935 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 09:25:05.437935 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:25:05.437935 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 27 09:25:05.437935 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:25:05.438666 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 366/743] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Oct 27 09:25:05 2022] GET /identity/v3/services/compute => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:25:05.444660 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25936b1c-6249-4b5d-ae2e-65cc018063d3 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:05.446576 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:05.460365 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e12acbd-ed48-4be9-acbf-0531f7f91b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:05.461331 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12acbd-ed48-4be9-acbf-0531f7f91b28 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:05.461607 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12acbd-ed48-4be9-acbf-0531f7f91b28 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:05.461873 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12acbd-ed48-4be9-acbf-0531f7f91b28 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:05.462796 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e12acbd-ed48-4be9-acbf-0531f7f91b28 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:25:05.463101 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e12acbd-ed48-4be9-acbf-0531f7f91b28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:05.464727 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e12acbd-ed48-4be9-acbf-0531f7f91b28 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:05.470495 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 378/744] 10.222.0.70 () {56 vars in 1122 bytes} [Thu Oct 27 09:25:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:25:05.476306 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e12acbd-ed48-4be9-acbf-0531f7f91b28 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:05.511061 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:05.577154 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25936b1c-6249-4b5d-ae2e-65cc018063d3 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:05.610371 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e12acbd-ed48-4be9-acbf-0531f7f91b28 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:05.613096 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88953693-5eb9-49d7-8171-f5c734cbbb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:05.613701 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88953693-5eb9-49d7-8171-f5c734cbbb06 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:05.613978 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88953693-5eb9-49d7-8171-f5c734cbbb06 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:05.614247 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88953693-5eb9-49d7-8171-f5c734cbbb06 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:05.615406 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88953693-5eb9-49d7-8171-f5c734cbbb06 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:25:05.615703 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88953693-5eb9-49d7-8171-f5c734cbbb06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:05.617162 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88953693-5eb9-49d7-8171-f5c734cbbb06 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:05.622354 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 367/745] 10.222.0.70 () {56 vars in 1112 bytes} [Thu Oct 27 09:25:05 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:25:05.627460 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88953693-5eb9-49d7-8171-f5c734cbbb06 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:05.657016 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b63a09-271f-4df3-9b5d-e30fe531c313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:05.658267 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b63a09-271f-4df3-9b5d-e30fe531c313 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:05.658730 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b63a09-271f-4df3-9b5d-e30fe531c313 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:05.659408 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b63a09-271f-4df3-9b5d-e30fe531c313 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:05.660209 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b63a09-271f-4df3-9b5d-e30fe531c313 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:25:05.660751 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b63a09-271f-4df3-9b5d-e30fe531c313 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:05.663320 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b63a09-271f-4df3-9b5d-e30fe531c313 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:05.669753 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 379/746] 10.222.0.70 () {56 vars in 1122 bytes} [Thu Oct 27 09:25:05 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:25:05.679023 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b63a09-271f-4df3-9b5d-e30fe531c313 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:05.743402 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:05.745722 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88953693-5eb9-49d7-8171-f5c734cbbb06 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:05.760097 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:05.785634 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f99f32-6ec4-4e42-a083-1cbf38a01ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:05.787065 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f99f32-6ec4-4e42-a083-1cbf38a01ff9 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:05.787530 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f99f32-6ec4-4e42-a083-1cbf38a01ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:05.787974 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f99f32-6ec4-4e42-a083-1cbf38a01ff9 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:05.788797 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f99f32-6ec4-4e42-a083-1cbf38a01ff9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:25:05.789274 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f99f32-6ec4-4e42-a083-1cbf38a01ff9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:05.790856 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f99f32-6ec4-4e42-a083-1cbf38a01ff9 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:05.796628 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 368/747] 10.222.0.70 () {56 vars in 1112 bytes} [Thu Oct 27 09:25:05 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:25:05.802783 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f99f32-6ec4-4e42-a083-1cbf38a01ff9 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:05.812155 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b63a09-271f-4df3-9b5d-e30fe531c313 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:05.850220 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832d5176-49fa-469f-ab75-9db3f1c8dc7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:05.850891 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832d5176-49fa-469f-ab75-9db3f1c8dc7c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:05.851078 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832d5176-49fa-469f-ab75-9db3f1c8dc7c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:05.851377 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832d5176-49fa-469f-ab75-9db3f1c8dc7c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:05.852070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832d5176-49fa-469f-ab75-9db3f1c8dc7c None admin] RBAC: Adding query filter params (interface=admin, service_id=c2231275295e48e69d2ab04ec979ff0e, region_id=RegionOne) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:25:05.852377 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832d5176-49fa-469f-ab75-9db3f1c8dc7c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:05.853952 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832d5176-49fa-469f-ab75-9db3f1c8dc7c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:05.860259 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 380/748] 10.222.0.70 () {56 vars in 1249 bytes} [Thu Oct 27 09:25:05 2022] GET /identity/v3/endpoints?service_id=c2231275295e48e69d2ab04ec979ff0e&interface=admin®ion_id=RegionOne => generated 193 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:05.866462 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832d5176-49fa-469f-ab75-9db3f1c8dc7c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:05.915111 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f99f32-6ec4-4e42-a083-1cbf38a01ff9 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:05.961188 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653afe3e-6160-47da-94d0-829c5b00b8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:05.961721 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653afe3e-6160-47da-94d0-829c5b00b8e1 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:05.962004 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653afe3e-6160-47da-94d0-829c5b00b8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:05.962278 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653afe3e-6160-47da-94d0-829c5b00b8e1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:05.962976 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653afe3e-6160-47da-94d0-829c5b00b8e1 None admin] RBAC: Adding query filter params (interface=public, service_id=122427e53cc740cb8fccef9f7e183832, region_id=RegionOne) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:25:05.963284 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653afe3e-6160-47da-94d0-829c5b00b8e1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:05.964683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653afe3e-6160-47da-94d0-829c5b00b8e1 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:05.972428 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 369/749] 10.222.0.70 () {56 vars in 1251 bytes} [Thu Oct 27 09:25:05 2022] GET /identity/v3/endpoints?service_id=122427e53cc740cb8fccef9f7e183832&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:05.978212 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653afe3e-6160-47da-94d0-829c5b00b8e1 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:05.997038 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832d5176-49fa-469f-ab75-9db3f1c8dc7c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:06.017700 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:06.038470 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:06.041241 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fa6cee-0eca-4343-abb3-0d156ab0cc92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:06.042278 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa6cee-0eca-4343-abb3-0d156ab0cc92 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:06.042625 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa6cee-0eca-4343-abb3-0d156ab0cc92 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:06.043008 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa6cee-0eca-4343-abb3-0d156ab0cc92 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:06.043648 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fa6cee-0eca-4343-abb3-0d156ab0cc92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:06.046464 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fa6cee-0eca-4343-abb3-0d156ab0cc92 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:06.055626 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 381/750] 10.222.0.70 () {56 vars in 1087 bytes} [Thu Oct 27 09:25:05 2022] GET /identity/v3/services => generated 1727 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:25:06.115715 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653afe3e-6160-47da-94d0-829c5b00b8e1 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:06.155541 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f839b0-0359-4727-9425-8f138de0ef36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:06.156502 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f839b0-0359-4727-9425-8f138de0ef36 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:06.156965 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f839b0-0359-4727-9425-8f138de0ef36 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:06.157400 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f839b0-0359-4727-9425-8f138de0ef36 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:06.158188 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f839b0-0359-4727-9425-8f138de0ef36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:06.159753 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f839b0-0359-4727-9425-8f138de0ef36 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:06.168027 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 370/751] 10.222.0.70 () {56 vars in 1087 bytes} [Thu Oct 27 09:25:05 2022] GET /identity/v3/services => generated 1727 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 09:25:06.346063 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53262ed-4a9c-4f2a-a71e-5d23bab9f931 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:06.346474 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53262ed-4a9c-4f2a-a71e-5d23bab9f931 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:06.346597 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53262ed-4a9c-4f2a-a71e-5d23bab9f931 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:06.348089 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 382/752] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:25:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:25:06.355129 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e39a52-7626-4923-b987-ba3257864a4f None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:06.355411 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e39a52-7626-4923-b987-ba3257864a4f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:06.355678 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e39a52-7626-4923-b987-ba3257864a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:06.374654 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-55e39a52-7626-4923-b987-ba3257864a4f None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:25:06.376194 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-55e39a52-7626-4923-b987-ba3257864a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:25:06.449373 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 371/753] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:25:06 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 27 09:25:06.724009 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53262ed-4a9c-4f2a-a71e-5d23bab9f931 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:06.834313 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53262ed-4a9c-4f2a-a71e-5d23bab9f931 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:06.871107 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69a6b44-ad71-449e-8686-138079115627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:06.872510 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a6b44-ad71-449e-8686-138079115627 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:06.873037 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a6b44-ad71-449e-8686-138079115627 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:06.873505 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a6b44-ad71-449e-8686-138079115627 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:06.874359 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69a6b44-ad71-449e-8686-138079115627 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:06.875913 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69a6b44-ad71-449e-8686-138079115627 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:06.896672 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 383/754] 10.222.0.70 () {60 vars in 1141 bytes} [Thu Oct 27 09:25:06 2022] POST /identity/v3/services => generated 242 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:25:07.264577 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2389eb-8017-44e8-8ea1-1a10ed0cd5d2 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:07.265113 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2389eb-8017-44e8-8ea1-1a10ed0cd5d2 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:07.265113 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2389eb-8017-44e8-8ea1-1a10ed0cd5d2 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:07.266822 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 372/755] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:25:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:25:07.274694 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0093e6-3f55-437d-95d7-13c0a69981fd None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:07.274983 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0093e6-3f55-437d-95d7-13c0a69981fd None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:07.275255 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0093e6-3f55-437d-95d7-13c0a69981fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:07.295347 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-1d0093e6-3f55-437d-95d7-13c0a69981fd None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:25:07.296908 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1d0093e6-3f55-437d-95d7-13c0a69981fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:25:07.333215 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:07.380631 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 384/756] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:25:07 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 09:25:07.644578 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2389eb-8017-44e8-8ea1-1a10ed0cd5d2 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:07.677332 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:07.752109 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2389eb-8017-44e8-8ea1-1a10ed0cd5d2 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:07.788779 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2ad023-888d-42a7-80bc-fd6c148e96b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:07.789384 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ad023-888d-42a7-80bc-fd6c148e96b2 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:07.789669 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ad023-888d-42a7-80bc-fd6c148e96b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:07.789946 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ad023-888d-42a7-80bc-fd6c148e96b2 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:07.790637 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2ad023-888d-42a7-80bc-fd6c148e96b2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:07.792132 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2ad023-888d-42a7-80bc-fd6c148e96b2 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:07.794690 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: WARNING keystone.server.flask.common [None req-ee2ad023-888d-42a7-80bc-fd6c148e96b2 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 27 09:25:07.809688 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.sql.core [None req-ee2ad023-888d-42a7-80bc-fd6c148e96b2 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 27 09:25:07.809688 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 27 09:25:07.809688 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [parameters: {'user_id': '5c75fb006f2f4ad1a08872a76f062cc4', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 27 09:25:07.809688 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81437) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 27 09:25:07.817320 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-ee2ad023-888d-42a7-80bc-fd6c148e96b2 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 27 09:25:07.817320 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:07.817320 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 27 09:25:07.817320 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 27 09:25:07.817320 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 27 09:25:07.817320 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 27 09:25:07.817320 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 27 09:25:07.817320 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application result = self._query(query) Oct 27 09:25:07.817320 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 27 09:25:07.817320 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application conn.query(q) Oct 27 09:25:07.817320 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 27 09:25:07.817320 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 27 09:25:07.817320 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 27 09:25:07.817320 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application result.read() Oct 27 09:25:07.817320 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 27 09:25:07.818243 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 27 09:25:07.818243 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 27 09:25:07.818243 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 27 09:25:07.818243 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 27 09:25:07.818243 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 27 09:25:07.818243 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 27 09:25:07.818243 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 27 09:25:07.818243 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 27 09:25:07.818243 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:25:07.818243 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 27 09:25:07.818243 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:25:07.818243 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:07.818243 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 27 09:25:07.818243 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 27 09:25:07.818243 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 27 09:25:07.818243 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 27 09:25:07.818243 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 27 09:25:07.818243 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application next(self.gen) Oct 27 09:25:07.819177 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 27 09:25:07.819177 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application yield resource Oct 27 09:25:07.819177 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 27 09:25:07.819177 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application next(self.gen) Oct 27 09:25:07.819177 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 27 09:25:07.819177 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self.session.rollback() Oct 27 09:25:07.819177 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 27 09:25:07.819177 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self.force_reraise() Oct 27 09:25:07.819177 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 27 09:25:07.819177 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise self.value Oct 27 09:25:07.819177 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 27 09:25:07.819177 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 27 09:25:07.819177 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 27 09:25:07.819177 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application session.commit() Oct 27 09:25:07.819177 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 27 09:25:07.819177 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 27 09:25:07.819177 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 27 09:25:07.820039 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._prepare_impl() Oct 27 09:25:07.820039 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 27 09:25:07.820039 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self.session.flush() Oct 27 09:25:07.820039 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 27 09:25:07.820039 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._flush(objects) Oct 27 09:25:07.820039 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 27 09:25:07.820039 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 27 09:25:07.820039 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 27 09:25:07.820039 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application compat.raise_( Oct 27 09:25:07.820039 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 27 09:25:07.820039 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception Oct 27 09:25:07.820039 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 27 09:25:07.820039 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application flush_context.execute() Oct 27 09:25:07.820039 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 27 09:25:07.820039 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rec.execute(self) Oct 27 09:25:07.820039 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 27 09:25:07.820039 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 27 09:25:07.820985 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 27 09:25:07.820985 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 27 09:25:07.820985 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 27 09:25:07.820985 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 27 09:25:07.820985 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 27 09:25:07.820985 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 27 09:25:07.820985 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 27 09:25:07.820985 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 27 09:25:07.820985 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 27 09:25:07.820985 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 27 09:25:07.820985 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 27 09:25:07.820985 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 27 09:25:07.820985 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 27 09:25:07.820985 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 27 09:25:07.820985 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 27 09:25:07.820985 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception Oct 27 09:25:07.821853 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 27 09:25:07.821853 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 27 09:25:07.821853 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 27 09:25:07.821853 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 27 09:25:07.821853 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 27 09:25:07.821853 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application result = self._query(query) Oct 27 09:25:07.821853 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 27 09:25:07.821853 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application conn.query(q) Oct 27 09:25:07.821853 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 27 09:25:07.821853 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 27 09:25:07.821853 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 27 09:25:07.821853 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application result.read() Oct 27 09:25:07.821853 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 27 09:25:07.821853 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 27 09:25:07.821853 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 27 09:25:07.821853 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 27 09:25:07.822659 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 27 09:25:07.822659 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 27 09:25:07.822659 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 27 09:25:07.822659 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 27 09:25:07.822659 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 27 09:25:07.822659 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 27 09:25:07.822659 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [parameters: {'user_id': '5c75fb006f2f4ad1a08872a76f062cc4', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 27 09:25:07.822659 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 27 09:25:07.822659 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:25:07.822659 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 09:25:07.822659 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:25:07.822659 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:07.822659 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:25:07.822659 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:25:07.822659 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:25:07.823531 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:25:07.823531 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:25:07.823531 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:25:07.823531 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:25:07.823531 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:25:07.823531 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:25:07.823531 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:25:07.823531 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 27 09:25:07.823531 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 27 09:25:07.823531 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:25:07.823531 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:25:07.823531 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 09:25:07.823531 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:25:07.823531 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 09:25:07.823531 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:25:07.823531 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 27 09:25:07.823531 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 27 09:25:07.826489 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 27 09:25:07.826489 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 27 09:25:07.826489 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 27 09:25:07.826489 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 27 09:25:07.826489 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 27 09:25:07.826489 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:25:07.826489 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 373/757] 10.222.0.70 () {60 vars in 1131 bytes} [Thu Oct 27 09:25:07 2022] POST /identity/v3/users => generated 160 bytes in 175 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:25:07.827386 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0093e6-3f55-437d-95d7-13c0a69981fd None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:07.933804 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0093e6-3f55-437d-95d7-13c0a69981fd None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:07.969051 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6f2a95-7984-401e-b0ef-374514ebdb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:07.969682 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6f2a95-7984-401e-b0ef-374514ebdb9d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:07.969959 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6f2a95-7984-401e-b0ef-374514ebdb9d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:07.970231 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6f2a95-7984-401e-b0ef-374514ebdb9d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:07.977938 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6f2a95-7984-401e-b0ef-374514ebdb9d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:07.979382 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6f2a95-7984-401e-b0ef-374514ebdb9d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:07.991584 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-de6f2a95-7984-401e-b0ef-374514ebdb9d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 27 09:25:07.991584 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:07.991584 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:25:07.991584 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:25:07.991584 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:25:07.991584 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:25:07.991584 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:25:07.991584 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:25:07.991584 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:25:07.991584 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:25:07.991584 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:25:07.991584 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:25:07.991584 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 09:25:07.991584 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 09:25:07.991584 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 09:25:07.991584 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 09:25:07.993816 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:25:07.993816 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:25:07.993816 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 09:25:07.993816 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:25:07.993816 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 09:25:07.993816 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:25:07.993816 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:25:07.993816 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:25:07.993816 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:25:07.993816 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:25:07.993816 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:25:07.993816 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:25:07.993816 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:25:07.993816 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:25:07.993816 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:25:07.993816 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:25:07.993816 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:25:07.993816 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:25:07.996075 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:25:07.996075 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:25:07.996075 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 09:25:07.996075 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 09:25:07.996075 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 09:25:07.996075 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 09:25:07.996075 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 09:25:07.996075 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 09:25:07.996075 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 27 09:25:07.996075 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:25:07.997376 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 385/758] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Oct 27 09:25:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:25:08.000574 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2ad023-888d-42a7-80bc-fd6c148e96b2 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:08.096902 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:08.104263 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2ad023-888d-42a7-80bc-fd6c148e96b2 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:08.140108 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc96cff-3dff-4afc-b07f-0026bccf0034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:08.140740 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc96cff-3dff-4afc-b07f-0026bccf0034 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:08.141020 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc96cff-3dff-4afc-b07f-0026bccf0034 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:08.141293 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc96cff-3dff-4afc-b07f-0026bccf0034 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:08.141993 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc96cff-3dff-4afc-b07f-0026bccf0034 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:25:08.142296 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc96cff-3dff-4afc-b07f-0026bccf0034 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:08.143977 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc96cff-3dff-4afc-b07f-0026bccf0034 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:08.187642 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 374/759] 10.222.0.70 () {56 vars in 1101 bytes} [Thu Oct 27 09:25:07 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:08.278686 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f676f73-6322-4025-98d0-4312e414d6c8 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:08.279000 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f676f73-6322-4025-98d0-4312e414d6c8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:08.279242 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f676f73-6322-4025-98d0-4312e414d6c8 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:08.282445 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 386/760] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:25:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:25:08.291194 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188ca05c-339e-494d-a072-0e28012fb269 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:08.291738 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188ca05c-339e-494d-a072-0e28012fb269 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:08.291738 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188ca05c-339e-494d-a072-0e28012fb269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:08.312118 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-188ca05c-339e-494d-a072-0e28012fb269 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:25:08.313788 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-188ca05c-339e-494d-a072-0e28012fb269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:25:08.392180 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 375/761] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:25:08 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 09:25:08.442344 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624403ec-8f26-4be7-8f54-99989d35f99a None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:08.442557 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624403ec-8f26-4be7-8f54-99989d35f99a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:08.442893 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624403ec-8f26-4be7-8f54-99989d35f99a None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:08.444493 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 387/762] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:25:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:25:08.449697 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6e2b34-c10d-47f4-b4ef-f209a9858d81 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:08.449990 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6e2b34-c10d-47f4-b4ef-f209a9858d81 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:08.450269 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6e2b34-c10d-47f4-b4ef-f209a9858d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:08.470233 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-8c6e2b34-c10d-47f4-b4ef-f209a9858d81 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:25:08.471416 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8c6e2b34-c10d-47f4-b4ef-f209a9858d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:25:08.544559 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 376/763] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:25:08 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 09:25:08.662627 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624403ec-8f26-4be7-8f54-99989d35f99a None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:08.690687 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:08.766593 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624403ec-8f26-4be7-8f54-99989d35f99a None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:08.799723 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905d06b5-bec8-4bc1-baa0-cbde2452d019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:08.800345 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905d06b5-bec8-4bc1-baa0-cbde2452d019 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:08.800653 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905d06b5-bec8-4bc1-baa0-cbde2452d019 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:08.800926 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905d06b5-bec8-4bc1-baa0-cbde2452d019 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:08.801496 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905d06b5-bec8-4bc1-baa0-cbde2452d019 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:08.802884 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905d06b5-bec8-4bc1-baa0-cbde2452d019 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:08.809950 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6e2b34-c10d-47f4-b4ef-f209a9858d81 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:08.812054 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-905d06b5-bec8-4bc1-baa0-cbde2452d019 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 27 09:25:08.812054 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:08.812054 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 09:25:08.812054 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 09:25:08.812054 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 09:25:08.812054 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 09:25:08.812054 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 09:25:08.812054 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:25:08.812054 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 27 09:25:08.812054 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:25:08.812054 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 09:25:08.812054 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:25:08.812054 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:08.812054 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:25:08.812054 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:25:08.812054 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:25:08.812054 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:25:08.816907 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:25:08.816907 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:25:08.816907 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:25:08.816907 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:25:08.816907 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:25:08.816907 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:25:08.816907 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 09:25:08.816907 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 09:25:08.816907 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 09:25:08.816907 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 09:25:08.816907 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:25:08.816907 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:25:08.816907 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:25:08.816907 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:25:08.816907 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:25:08.816907 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:25:08.818589 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:25:08.818589 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:25:08.818589 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:25:08.818589 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:25:08.818589 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:25:08.818589 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:25:08.818589 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:25:08.818589 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:25:08.818589 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:25:08.818589 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:25:08.818589 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 09:25:08.818589 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:25:08.818589 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 27 09:25:08.818589 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:25:08.818589 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 388/764] 10.222.0.70 () {56 vars in 1126 bytes} [Thu Oct 27 09:25:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 09:25:08.820396 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905d06b5-bec8-4bc1-baa0-cbde2452d019 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:08.825032 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:08.919388 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6e2b34-c10d-47f4-b4ef-f209a9858d81 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:08.923908 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:08.934406 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905d06b5-bec8-4bc1-baa0-cbde2452d019 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:08.957279 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cef8ac6-70cf-4312-a293-97f9290725da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:08.958017 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef8ac6-70cf-4312-a293-97f9290725da None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:08.958854 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef8ac6-70cf-4312-a293-97f9290725da None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:08.959237 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef8ac6-70cf-4312-a293-97f9290725da None admin] PATH_INFO: `/v3/services/compute` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:08.959865 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cef8ac6-70cf-4312-a293-97f9290725da None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:08.961384 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cef8ac6-70cf-4312-a293-97f9290725da None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:08.970764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-3cef8ac6-70cf-4312-a293-97f9290725da None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 27 09:25:08.970764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:08.970764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 09:25:08.970764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 09:25:08.970764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 09:25:08.970764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 09:25:08.970764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 09:25:08.970764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:25:08.970764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 27 09:25:08.970764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:25:08.970764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 09:25:08.970764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:25:08.970764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:08.970764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:25:08.970764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:25:08.970764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:25:08.970764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:25:08.971739 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:25:08.971739 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:25:08.971739 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:25:08.971739 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:25:08.971739 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:25:08.971739 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:25:08.971739 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 09:25:08.971739 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 09:25:08.971739 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 09:25:08.971739 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 09:25:08.971739 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:25:08.971739 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:25:08.971739 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:25:08.971739 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:25:08.971739 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:25:08.971739 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:25:08.973436 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:25:08.973436 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:25:08.973436 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:25:08.973436 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:25:08.973436 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:25:08.973436 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:25:08.973436 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:25:08.973436 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:25:08.973436 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:25:08.973436 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:25:08.973436 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 09:25:08.973436 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:25:08.973436 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 27 09:25:08.973436 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:25:08.974200 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2522f157-f8d4-4177-bb8b-517cdd56941c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:08.974200 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2522f157-f8d4-4177-bb8b-517cdd56941c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:08.974200 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2522f157-f8d4-4177-bb8b-517cdd56941c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:08.974412 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 377/765] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Oct 27 09:25:08 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 09:25:08.974481 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2522f157-f8d4-4177-bb8b-517cdd56941c None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:08.974481 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2522f157-f8d4-4177-bb8b-517cdd56941c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:25:08.974481 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2522f157-f8d4-4177-bb8b-517cdd56941c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:08.975517 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2522f157-f8d4-4177-bb8b-517cdd56941c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:08.977899 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cef8ac6-70cf-4312-a293-97f9290725da None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:08.980976 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 389/766] 10.222.0.70 () {56 vars in 1122 bytes} [Thu Oct 27 09:25:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:08.987175 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2522f157-f8d4-4177-bb8b-517cdd56941c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:09.063954 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:09.087904 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cef8ac6-70cf-4312-a293-97f9290725da None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:09.098471 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2522f157-f8d4-4177-bb8b-517cdd56941c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:09.129164 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b900d44-966b-4729-8a78-a674f1aaf203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:09.129783 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b900d44-966b-4729-8a78-a674f1aaf203 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:09.130640 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b900d44-966b-4729-8a78-a674f1aaf203 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:09.130921 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b900d44-966b-4729-8a78-a674f1aaf203 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:09.131550 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b900d44-966b-4729-8a78-a674f1aaf203 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:25:09.131865 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b900d44-966b-4729-8a78-a674f1aaf203 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:09.133265 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b900d44-966b-4729-8a78-a674f1aaf203 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:09.138154 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6fca0e-0962-4d3e-b14b-8947c9358357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:09.138370 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 378/767] 10.222.0.70 () {56 vars in 1112 bytes} [Thu Oct 27 09:25:08 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:25:09.139121 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6fca0e-0962-4d3e-b14b-8947c9358357 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:09.139480 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6fca0e-0962-4d3e-b14b-8947c9358357 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:09.139754 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6fca0e-0962-4d3e-b14b-8947c9358357 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:09.140372 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6fca0e-0962-4d3e-b14b-8947c9358357 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:25:09.140703 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6fca0e-0962-4d3e-b14b-8947c9358357 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:09.142040 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6fca0e-0962-4d3e-b14b-8947c9358357 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:09.143946 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b900d44-966b-4729-8a78-a674f1aaf203 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:09.152269 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 390/768] 10.222.0.70 () {56 vars in 1122 bytes} [Thu Oct 27 09:25:08 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:25:09.153048 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6fca0e-0962-4d3e-b14b-8947c9358357 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:09.173044 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:09.247857 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b900d44-966b-4729-8a78-a674f1aaf203 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:09.256921 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6fca0e-0962-4d3e-b14b-8947c9358357 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:09.283411 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704f3ba9-bbed-4d6e-8ee7-e3bd5653ad67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:09.284025 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704f3ba9-bbed-4d6e-8ee7-e3bd5653ad67 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:09.284300 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704f3ba9-bbed-4d6e-8ee7-e3bd5653ad67 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:09.284753 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704f3ba9-bbed-4d6e-8ee7-e3bd5653ad67 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:09.285373 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704f3ba9-bbed-4d6e-8ee7-e3bd5653ad67 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:25:09.285674 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704f3ba9-bbed-4d6e-8ee7-e3bd5653ad67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:09.287065 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704f3ba9-bbed-4d6e-8ee7-e3bd5653ad67 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:09.291900 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 379/769] 10.222.0.70 () {56 vars in 1112 bytes} [Thu Oct 27 09:25:09 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:09.293073 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2480e5f6-470f-4fce-a67a-acff1d0bb816 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:09.293354 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2480e5f6-470f-4fce-a67a-acff1d0bb816 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:09.293625 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2480e5f6-470f-4fce-a67a-acff1d0bb816 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:09.294803 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 380/770] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:25:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:25:09.297409 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2480e5f6-470f-4fce-a67a-acff1d0bb816 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:09.299607 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c8c654-cc89-4487-a246-760c92286fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:09.300227 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c8c654-cc89-4487-a246-760c92286fc3 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:09.300531 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c8c654-cc89-4487-a246-760c92286fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:09.300813 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c8c654-cc89-4487-a246-760c92286fc3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:09.301477 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c8c654-cc89-4487-a246-760c92286fc3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:09.302650 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:09.303773 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c8c654-cc89-4487-a246-760c92286fc3 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:09.320541 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 391/771] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:25:09 2022] POST /identity/v3/endpoints => generated 325 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:25:09.321745 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ada258-3600-4515-a67b-3f8fa65f2bf6 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:09.322036 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ada258-3600-4515-a67b-3f8fa65f2bf6 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:09.322306 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ada258-3600-4515-a67b-3f8fa65f2bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:09.351590 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-b1ada258-3600-4515-a67b-3f8fa65f2bf6 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:25:09.352995 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b1ada258-3600-4515-a67b-3f8fa65f2bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:25:09.420387 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2480e5f6-470f-4fce-a67a-acff1d0bb816 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:09.428122 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 392/772] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:25:09 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 09:25:09.472153 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78879db-a7eb-4bc3-a062-1ec53bbccf4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:09.472825 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78879db-a7eb-4bc3-a062-1ec53bbccf4c None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:09.473106 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78879db-a7eb-4bc3-a062-1ec53bbccf4c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:09.473377 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78879db-a7eb-4bc3-a062-1ec53bbccf4c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:09.474610 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78879db-a7eb-4bc3-a062-1ec53bbccf4c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:09.476067 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78879db-a7eb-4bc3-a062-1ec53bbccf4c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:09.499270 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 381/773] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:25:09 2022] POST /identity/v3/endpoints => generated 334 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:25:09.701470 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ada258-3600-4515-a67b-3f8fa65f2bf6 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:09.711903 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:09.807235 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ada258-3600-4515-a67b-3f8fa65f2bf6 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:09.850320 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa0cb1e-b621-4925-a0b1-3046e330212e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:09.850958 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0cb1e-b621-4925-a0b1-3046e330212e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:09.851225 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0cb1e-b621-4925-a0b1-3046e330212e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:09.851496 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0cb1e-b621-4925-a0b1-3046e330212e None admin] PATH_INFO: `/v3/services/image` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:09.852062 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa0cb1e-b621-4925-a0b1-3046e330212e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:09.853447 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa0cb1e-b621-4925-a0b1-3046e330212e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:09.862332 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-efa0cb1e-b621-4925-a0b1-3046e330212e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 27 09:25:09.862332 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:09.862332 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 09:25:09.862332 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 09:25:09.862332 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 09:25:09.862332 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 09:25:09.862332 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 09:25:09.862332 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:25:09.862332 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 27 09:25:09.862332 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:25:09.862332 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 09:25:09.862332 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:25:09.862332 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:09.862332 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:25:09.862332 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:25:09.862332 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:25:09.862332 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:25:09.863308 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:25:09.863308 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:25:09.863308 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:25:09.863308 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:25:09.863308 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:25:09.863308 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:25:09.863308 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 09:25:09.863308 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 09:25:09.863308 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 09:25:09.863308 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 09:25:09.863308 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:25:09.863308 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:25:09.863308 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:25:09.863308 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:25:09.863308 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:25:09.863308 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:25:09.864167 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:25:09.864167 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:25:09.864167 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:25:09.864167 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:25:09.864167 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:25:09.864167 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:25:09.864167 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:25:09.864167 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:25:09.864167 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:25:09.864167 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:25:09.864167 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 09:25:09.864167 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:25:09.864167 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 27 09:25:09.864167 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:25:09.868399 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 393/774] 10.222.0.70 () {56 vars in 1105 bytes} [Thu Oct 27 09:25:09 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:25:09.869802 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78879db-a7eb-4bc3-a062-1ec53bbccf4c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:09.941286 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:09.998694 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78879db-a7eb-4bc3-a062-1ec53bbccf4c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:10.032952 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0322c71-8b5f-4b11-9594-7dd5104a7b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:10.033556 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0322c71-8b5f-4b11-9594-7dd5104a7b9e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:10.033827 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0322c71-8b5f-4b11-9594-7dd5104a7b9e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:10.034090 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0322c71-8b5f-4b11-9594-7dd5104a7b9e None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:10.034703 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0322c71-8b5f-4b11-9594-7dd5104a7b9e None admin] RBAC: Adding query filter params (name=image) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:25:10.034997 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0322c71-8b5f-4b11-9594-7dd5104a7b9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:10.037010 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0322c71-8b5f-4b11-9594-7dd5104a7b9e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:10.041991 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 382/775] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:25:09 2022] GET /identity/v3/services?name=image => generated 122 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:10.047577 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa0cb1e-b621-4925-a0b1-3046e330212e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:10.117162 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:10.152056 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa0cb1e-b621-4925-a0b1-3046e330212e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:10.185387 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd45aaac-bf59-49dd-bdc4-b49ef62f5fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:10.185993 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45aaac-bf59-49dd-bdc4-b49ef62f5fae None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:10.186270 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45aaac-bf59-49dd-bdc4-b49ef62f5fae None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:10.186539 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45aaac-bf59-49dd-bdc4-b49ef62f5fae None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:10.187154 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd45aaac-bf59-49dd-bdc4-b49ef62f5fae None admin] RBAC: Adding query filter params (type=image) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:25:10.187453 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd45aaac-bf59-49dd-bdc4-b49ef62f5fae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:10.188832 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd45aaac-bf59-49dd-bdc4-b49ef62f5fae None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:10.194303 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 394/776] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:25:10 2022] GET /identity/v3/services?type=image => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:25:10.199938 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0322c71-8b5f-4b11-9594-7dd5104a7b9e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:10.306612 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0322c71-8b5f-4b11-9594-7dd5104a7b9e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:10.328467 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:10.344266 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e12c95-c701-4a37-ad53-581802a0cab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:10.344895 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e12c95-c701-4a37-ad53-581802a0cab6 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:10.345174 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e12c95-c701-4a37-ad53-581802a0cab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:10.345437 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e12c95-c701-4a37-ad53-581802a0cab6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:10.346137 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e12c95-c701-4a37-ad53-581802a0cab6 None admin] RBAC: Adding query filter params (interface=public, service_id=c3f6950f5b524898893e7e0aa94597e1, region_id=RegionOne) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:25:10.346437 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e12c95-c701-4a37-ad53-581802a0cab6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:10.347843 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e12c95-c701-4a37-ad53-581802a0cab6 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:10.353239 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 383/777] 10.222.0.70 () {56 vars in 1251 bytes} [Thu Oct 27 09:25:10 2022] GET /identity/v3/endpoints?service_id=c3f6950f5b524898893e7e0aa94597e1&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:10.358826 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd45aaac-bf59-49dd-bdc4-b49ef62f5fae None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:10.458666 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd45aaac-bf59-49dd-bdc4-b49ef62f5fae None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:10.486964 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:10.493705 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05260eb7-4089-4542-bd4b-1e1d498d65f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:10.494327 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05260eb7-4089-4542-bd4b-1e1d498d65f2 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:10.494599 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05260eb7-4089-4542-bd4b-1e1d498d65f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:10.494866 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05260eb7-4089-4542-bd4b-1e1d498d65f2 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:10.495470 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05260eb7-4089-4542-bd4b-1e1d498d65f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:10.496888 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05260eb7-4089-4542-bd4b-1e1d498d65f2 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:10.504180 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 395/778] 10.222.0.70 () {56 vars in 1087 bytes} [Thu Oct 27 09:25:10 2022] GET /identity/v3/services => generated 1957 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:25:11.531874 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde60c0-3ffa-4864-b128-6c7bc61aa9d2 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:11.532387 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde60c0-3ffa-4864-b128-6c7bc61aa9d2 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:11.532387 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde60c0-3ffa-4864-b128-6c7bc61aa9d2 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:11.533981 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 384/779] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:25:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:25:11.538879 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae5a727-2fe9-47c0-a942-33795abcfeb8 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:11.539253 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae5a727-2fe9-47c0-a942-33795abcfeb8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:11.539412 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae5a727-2fe9-47c0-a942-33795abcfeb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:11.557674 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-bae5a727-2fe9-47c0-a942-33795abcfeb8 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:25:11.558839 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-bae5a727-2fe9-47c0-a942-33795abcfeb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:25:11.626266 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 396/780] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:25:11 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 09:25:11.784352 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327aaf7-b801-4d8b-aa38-8b134882d10f None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:11.784726 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327aaf7-b801-4d8b-aa38-8b134882d10f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:11.784825 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327aaf7-b801-4d8b-aa38-8b134882d10f None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:11.786365 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 385/781] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:25:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:25:11.791324 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e3fd9-ab35-4893-90b2-d4d49ffe7474 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:11.791611 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e3fd9-ab35-4893-90b2-d4d49ffe7474 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:11.791884 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e3fd9-ab35-4893-90b2-d4d49ffe7474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:11.810642 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-7c7e3fd9-ab35-4893-90b2-d4d49ffe7474 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:25:11.811818 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7c7e3fd9-ab35-4893-90b2-d4d49ffe7474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:25:11.877808 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 397/782] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:25:11 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 09:25:11.893473 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4327aaf7-b801-4d8b-aa38-8b134882d10f None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:11.993665 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4327aaf7-b801-4d8b-aa38-8b134882d10f None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:12.027255 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cafc3e8-669f-40ef-bd63-7203cbd2ce4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:12.027874 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cafc3e8-669f-40ef-bd63-7203cbd2ce4c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:12.028150 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cafc3e8-669f-40ef-bd63-7203cbd2ce4c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:12.028417 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cafc3e8-669f-40ef-bd63-7203cbd2ce4c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:12.030131 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cafc3e8-669f-40ef-bd63-7203cbd2ce4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:12.031555 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cafc3e8-669f-40ef-bd63-7203cbd2ce4c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:12.033336 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 386/783] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:25:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:12.038877 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e3fd9-ab35-4893-90b2-d4d49ffe7474 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:12.095490 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:12.140138 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e3fd9-ab35-4893-90b2-d4d49ffe7474 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:12.150880 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cafc3e8-669f-40ef-bd63-7203cbd2ce4c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:12.174585 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00518ff-cd44-4d55-9022-e8d1ef27bc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:12.176185 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00518ff-cd44-4d55-9022-e8d1ef27bc3b None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:12.176185 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00518ff-cd44-4d55-9022-e8d1ef27bc3b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:12.176185 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00518ff-cd44-4d55-9022-e8d1ef27bc3b None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:12.176545 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00518ff-cd44-4d55-9022-e8d1ef27bc3b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:12.178517 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00518ff-cd44-4d55-9022-e8d1ef27bc3b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:12.206512 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:12.208393 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 398/784] 10.222.0.70 () {60 vars in 1141 bytes} [Thu Oct 27 09:25:12 2022] POST /identity/v3/projects => generated 311 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:25:12.274937 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cafc3e8-669f-40ef-bd63-7203cbd2ce4c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:12.314159 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a765494b-e1ff-4bc1-ad25-fba10fd30f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:12.314723 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765494b-e1ff-4bc1-ad25-fba10fd30f21 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:12.315003 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765494b-e1ff-4bc1-ad25-fba10fd30f21 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:12.315271 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765494b-e1ff-4bc1-ad25-fba10fd30f21 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:12.327851 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a765494b-e1ff-4bc1-ad25-fba10fd30f21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:25:12.333350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-a765494b-e1ff-4bc1-ad25-fba10fd30f21 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 27 09:25:12.333350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:12.333350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:25:12.333350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:25:12.333350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:25:12.333350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:25:12.333350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:25:12.333350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:25:12.333350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:25:12.333350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:25:12.333350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:25:12.333350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:25:12.333350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:25:12.333350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:25:12.333350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:25:12.333350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:25:12.334350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:25:12.334350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:25:12.334350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:25:12.334350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:25:12.334350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:25:12.334350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:25:12.334350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:25:12.334350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:25:12.334350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:25:12.334350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:25:12.334350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:25:12.334350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:25:12.334350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:25:12.334350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:25:12.334350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:25:12.334350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:25:12.335196 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:25:12.335196 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:25:12.335196 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:25:12.335196 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:25:12.335196 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:25:12.335196 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:25:12.335196 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:25:12.335196 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:25:12.335196 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:25:12.335196 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:25:12.335196 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 27 09:25:12.335196 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:25:12.336725 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 387/785] 10.222.0.70 () {56 vars in 1120 bytes} [Thu Oct 27 09:25:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 09:25:12.340743 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00518ff-cd44-4d55-9022-e8d1ef27bc3b None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:12.450281 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00518ff-cd44-4d55-9022-e8d1ef27bc3b None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:12.459555 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:12.485823 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc94c50-0db9-4a2e-9947-c567747425ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:12.486528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc94c50-0db9-4a2e-9947-c567747425ee None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:12.486809 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc94c50-0db9-4a2e-9947-c567747425ee None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:12.487077 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc94c50-0db9-4a2e-9947-c567747425ee None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:12.487727 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc94c50-0db9-4a2e-9947-c567747425ee None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:25:12.488028 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc94c50-0db9-4a2e-9947-c567747425ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:12.489984 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc94c50-0db9-4a2e-9947-c567747425ee None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:12.503005 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 399/786] 10.222.0.70 () {56 vars in 1115 bytes} [Thu Oct 27 09:25:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:12.504496 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a765494b-e1ff-4bc1-ad25-fba10fd30f21 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:12.609620 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a765494b-e1ff-4bc1-ad25-fba10fd30f21 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:12.623181 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:12.645749 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55f9d8-4295-4d01-a35a-48c4db0079af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:12.646420 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55f9d8-4295-4d01-a35a-48c4db0079af None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:12.646697 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55f9d8-4295-4d01-a35a-48c4db0079af None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:12.646966 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55f9d8-4295-4d01-a35a-48c4db0079af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:12.657841 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f55f9d8-4295-4d01-a35a-48c4db0079af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:12.659297 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f55f9d8-4295-4d01-a35a-48c4db0079af None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:12.673887 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-9f55f9d8-4295-4d01-a35a-48c4db0079af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:25:12.673887 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:12.673887 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:25:12.673887 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:25:12.673887 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:25:12.673887 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:25:12.673887 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:25:12.673887 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:25:12.673887 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:25:12.673887 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:25:12.673887 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:25:12.673887 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:25:12.673887 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:25:12.673887 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:25:12.673887 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:25:12.673887 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:25:12.675120 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:25:12.675120 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:25:12.675120 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:25:12.675120 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:25:12.675120 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:25:12.675120 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:25:12.675120 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:25:12.675120 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:25:12.675120 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:25:12.675120 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:25:12.675120 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:25:12.675120 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:25:12.675120 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:25:12.675120 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:25:12.675120 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:25:12.675120 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:25:12.675120 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:25:12.678051 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:25:12.678051 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:25:12.678051 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:25:12.678051 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 388/787] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:25:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 09:25:12.679954 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1e393-69a5-4df8-90fe-88d6ef203e47 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:12.680393 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1e393-69a5-4df8-90fe-88d6ef203e47 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:12.680794 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1e393-69a5-4df8-90fe-88d6ef203e47 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:12.681403 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55f9d8-4295-4d01-a35a-48c4db0079af None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:12.682335 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 400/788] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:25:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:25:12.687322 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6829055-f331-4264-8fb4-e87ae09ae1a2 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:12.687605 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6829055-f331-4264-8fb4-e87ae09ae1a2 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:12.687874 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6829055-f331-4264-8fb4-e87ae09ae1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:12.705834 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-a6829055-f331-4264-8fb4-e87ae09ae1a2 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:25:12.707017 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a6829055-f331-4264-8fb4-e87ae09ae1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:25:12.783935 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 401/789] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:25:12 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 27 09:25:12.787066 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55f9d8-4295-4d01-a35a-48c4db0079af None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:12.822053 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7409e2-1dbe-4f1e-9fc8-274a1d4f535e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:12.822662 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7409e2-1dbe-4f1e-9fc8-274a1d4f535e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:12.822940 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7409e2-1dbe-4f1e-9fc8-274a1d4f535e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:12.823209 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7409e2-1dbe-4f1e-9fc8-274a1d4f535e None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:12.823835 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7409e2-1dbe-4f1e-9fc8-274a1d4f535e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:25:12.824133 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7409e2-1dbe-4f1e-9fc8-274a1d4f535e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:12.825538 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7409e2-1dbe-4f1e-9fc8-274a1d4f535e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:12.959761 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 389/790] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:25:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:25:12.961485 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6829055-f331-4264-8fb4-e87ae09ae1a2 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:13.055899 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7409e2-1dbe-4f1e-9fc8-274a1d4f535e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:13.061191 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:13.065433 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6829055-f331-4264-8fb4-e87ae09ae1a2 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:13.082088 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:13.106810 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63040fe-34b2-4164-9cf9-0fd2840d69e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:13.107434 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63040fe-34b2-4164-9cf9-0fd2840d69e3 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:13.107715 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63040fe-34b2-4164-9cf9-0fd2840d69e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:13.107975 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63040fe-34b2-4164-9cf9-0fd2840d69e3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:13.115298 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63040fe-34b2-4164-9cf9-0fd2840d69e3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:13.116752 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63040fe-34b2-4164-9cf9-0fd2840d69e3 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:13.128559 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-e63040fe-34b2-4164-9cf9-0fd2840d69e3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 27 09:25:13.128559 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:13.128559 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:25:13.128559 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:25:13.128559 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:25:13.128559 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:25:13.128559 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:25:13.128559 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:25:13.128559 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:25:13.128559 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:25:13.128559 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:25:13.128559 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:25:13.128559 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 09:25:13.128559 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 09:25:13.128559 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 09:25:13.128559 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 09:25:13.129509 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:25:13.129509 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:25:13.129509 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 09:25:13.129509 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:25:13.129509 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 09:25:13.129509 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:25:13.129509 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:25:13.129509 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:25:13.129509 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:25:13.129509 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:25:13.129509 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:25:13.129509 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:25:13.129509 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:25:13.129509 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:25:13.129509 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:25:13.129509 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:25:13.129509 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:25:13.129509 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:25:13.130460 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:25:13.130460 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:25:13.130460 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 09:25:13.130460 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 09:25:13.130460 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 09:25:13.130460 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 09:25:13.130460 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 09:25:13.130460 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 09:25:13.130460 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 27 09:25:13.130460 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:25:13.134531 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 402/791] 10.222.0.70 () {56 vars in 1093 bytes} [Thu Oct 27 09:25:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 09:25:13.135726 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63040fe-34b2-4164-9cf9-0fd2840d69e3 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:13.170218 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7409e2-1dbe-4f1e-9fc8-274a1d4f535e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:13.209252 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da12bf44-4989-4ee7-a908-e0f9015b8c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:13.209935 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da12bf44-4989-4ee7-a908-e0f9015b8c16 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:13.210150 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da12bf44-4989-4ee7-a908-e0f9015b8c16 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:13.210410 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da12bf44-4989-4ee7-a908-e0f9015b8c16 None admin] PATH_INFO: `/v3/services/image` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:13.211048 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da12bf44-4989-4ee7-a908-e0f9015b8c16 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:13.212552 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da12bf44-4989-4ee7-a908-e0f9015b8c16 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:13.221435 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-da12bf44-4989-4ee7-a908-e0f9015b8c16 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 27 09:25:13.221435 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:13.221435 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 09:25:13.221435 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 09:25:13.221435 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 09:25:13.221435 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 09:25:13.221435 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 09:25:13.221435 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:25:13.221435 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 27 09:25:13.221435 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:25:13.221435 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 09:25:13.221435 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:25:13.221435 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:13.221435 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:25:13.221435 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:25:13.221435 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:25:13.221435 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:25:13.222427 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:25:13.222427 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:25:13.222427 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:25:13.222427 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:25:13.222427 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:25:13.222427 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:25:13.222427 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 09:25:13.222427 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 09:25:13.222427 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 09:25:13.222427 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 09:25:13.222427 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:25:13.222427 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:25:13.222427 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:25:13.222427 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:25:13.222427 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:25:13.222427 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:25:13.223285 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:25:13.223285 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:25:13.223285 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:25:13.223285 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:25:13.223285 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:25:13.223285 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:25:13.223285 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:25:13.223285 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:25:13.223285 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:25:13.223285 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:25:13.223285 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 09:25:13.223285 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:25:13.223285 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 27 09:25:13.223285 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:25:13.223285 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 390/792] 10.222.0.70 () {56 vars in 1105 bytes} [Thu Oct 27 09:25:13 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 09:25:13.228376 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da12bf44-4989-4ee7-a908-e0f9015b8c16 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:13.246105 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63040fe-34b2-4164-9cf9-0fd2840d69e3 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:13.289442 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322710a0-1826-4c2a-8b74-05615909c939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:13.290378 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322710a0-1826-4c2a-8b74-05615909c939 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:13.290792 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322710a0-1826-4c2a-8b74-05615909c939 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:13.291069 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322710a0-1826-4c2a-8b74-05615909c939 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:13.291790 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322710a0-1826-4c2a-8b74-05615909c939 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:25:13.292095 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322710a0-1826-4c2a-8b74-05615909c939 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:13.293541 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322710a0-1826-4c2a-8b74-05615909c939 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:13.310838 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 403/793] 10.222.0.70 () {56 vars in 1133 bytes} [Thu Oct 27 09:25:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:25:13.316488 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322710a0-1826-4c2a-8b74-05615909c939 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:13.328193 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:13.337252 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:13.342539 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da12bf44-4989-4ee7-a908-e0f9015b8c16 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:13.378581 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f868a50-d31b-4eb6-bba0-78a4ff11ee72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:13.379189 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f868a50-d31b-4eb6-bba0-78a4ff11ee72 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:13.379456 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f868a50-d31b-4eb6-bba0-78a4ff11ee72 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:13.380266 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f868a50-d31b-4eb6-bba0-78a4ff11ee72 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:13.380892 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f868a50-d31b-4eb6-bba0-78a4ff11ee72 None admin] RBAC: Adding query filter params (name=image) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:25:13.381186 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f868a50-d31b-4eb6-bba0-78a4ff11ee72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:13.382539 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f868a50-d31b-4eb6-bba0-78a4ff11ee72 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:13.392109 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 391/794] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:25:13 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:25:13.393029 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f868a50-d31b-4eb6-bba0-78a4ff11ee72 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:13.426153 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322710a0-1826-4c2a-8b74-05615909c939 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:13.462163 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c0fbce-f6fe-4a6e-8642-eaca0db04f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:13.462782 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0fbce-f6fe-4a6e-8642-eaca0db04f1c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:13.463066 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0fbce-f6fe-4a6e-8642-eaca0db04f1c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:13.463334 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0fbce-f6fe-4a6e-8642-eaca0db04f1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:13.474558 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c0fbce-f6fe-4a6e-8642-eaca0db04f1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:13.475988 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c0fbce-f6fe-4a6e-8642-eaca0db04f1c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:13.491111 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-21c0fbce-f6fe-4a6e-8642-eaca0db04f1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:25:13.491111 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:13.491111 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:25:13.491111 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:25:13.491111 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:25:13.491111 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:25:13.491111 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:25:13.491111 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:25:13.491111 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:25:13.491111 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:25:13.491111 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:25:13.491111 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:25:13.491111 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:25:13.491111 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:25:13.491111 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:25:13.491111 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:25:13.492065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:25:13.492065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:25:13.492065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:25:13.492065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:25:13.492065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:25:13.492065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:25:13.492065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:25:13.492065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:25:13.492065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:25:13.492065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:25:13.492065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:25:13.492065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:25:13.492065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:25:13.492065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:25:13.492065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:25:13.492065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:25:13.492065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:25:13.498185 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:25:13.498185 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:25:13.498185 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:25:13.498185 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 404/795] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:25:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 09:25:13.498715 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c0fbce-f6fe-4a6e-8642-eaca0db04f1c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:13.499753 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f868a50-d31b-4eb6-bba0-78a4ff11ee72 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:13.537768 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd29038c-1b37-4393-af11-3ed8ae3b5a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:13.538368 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd29038c-1b37-4393-af11-3ed8ae3b5a0a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:13.538637 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd29038c-1b37-4393-af11-3ed8ae3b5a0a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:13.538900 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd29038c-1b37-4393-af11-3ed8ae3b5a0a None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:13.539539 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd29038c-1b37-4393-af11-3ed8ae3b5a0a None admin] RBAC: Adding query filter params (type=image) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:25:13.539837 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd29038c-1b37-4393-af11-3ed8ae3b5a0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:13.541225 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd29038c-1b37-4393-af11-3ed8ae3b5a0a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:13.546269 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 392/796] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:25:13 2022] GET /identity/v3/services?type=image => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:13.551919 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd29038c-1b37-4393-af11-3ed8ae3b5a0a None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:13.571323 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:13.571708 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:13.608824 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c0fbce-f6fe-4a6e-8642-eaca0db04f1c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:13.645732 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb050d2-54d4-4083-b68c-d29037875580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:13.646333 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb050d2-54d4-4083-b68c-d29037875580 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:13.646612 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb050d2-54d4-4083-b68c-d29037875580 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:13.646883 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb050d2-54d4-4083-b68c-d29037875580 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:13.647672 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb050d2-54d4-4083-b68c-d29037875580 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:25:13.647846 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb050d2-54d4-4083-b68c-d29037875580 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:13.649239 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb050d2-54d4-4083-b68c-d29037875580 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:13.660138 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd29038c-1b37-4393-af11-3ed8ae3b5a0a None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:13.662650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 405/797] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:25:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:25:13.667980 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb050d2-54d4-4083-b68c-d29037875580 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:13.697269 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b821cb5a-c26c-41a1-93ce-869e0e026410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:13.697876 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b821cb5a-c26c-41a1-93ce-869e0e026410 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:13.698151 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b821cb5a-c26c-41a1-93ce-869e0e026410 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:13.698418 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b821cb5a-c26c-41a1-93ce-869e0e026410 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:13.699066 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b821cb5a-c26c-41a1-93ce-869e0e026410 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:13.700410 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b821cb5a-c26c-41a1-93ce-869e0e026410 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:13.720802 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 393/798] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:25:13 2022] POST /identity/v3/endpoints => generated 327 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:25:13.776159 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb050d2-54d4-4083-b68c-d29037875580 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:13.807761 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:13.822590 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e8f66b-052a-4613-9a7a-285ed6e16f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:13.823236 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8f66b-052a-4613-9a7a-285ed6e16f57 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:13.823511 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8f66b-052a-4613-9a7a-285ed6e16f57 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:13.823776 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8f66b-052a-4613-9a7a-285ed6e16f57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:13.831151 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e8f66b-052a-4613-9a7a-285ed6e16f57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:13.832643 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e8f66b-052a-4613-9a7a-285ed6e16f57 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:13.844996 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-90e8f66b-052a-4613-9a7a-285ed6e16f57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:25:13.844996 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:13.844996 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:25:13.844996 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:25:13.844996 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:25:13.844996 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:25:13.844996 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:25:13.844996 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:25:13.844996 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:25:13.844996 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:25:13.844996 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:25:13.844996 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:25:13.844996 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 09:25:13.844996 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 09:25:13.844996 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 09:25:13.845995 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 09:25:13.845995 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:25:13.845995 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:25:13.845995 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:25:13.845995 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:25:13.845995 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:25:13.845995 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:25:13.845995 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:25:13.845995 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:25:13.845995 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:25:13.845995 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:25:13.845995 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:25:13.845995 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:25:13.845995 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:25:13.845995 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:25:13.845995 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:25:13.845995 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:25:13.847425 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 09:25:13.847425 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 09:25:13.847425 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 09:25:13.847425 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 09:25:13.847425 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 09:25:13.847425 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 09:25:13.847425 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:25:13.847425 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:25:13.847425 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 406/799] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Oct 27 09:25:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 09:25:13.852138 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b821cb5a-c26c-41a1-93ce-869e0e026410 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:13.928894 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:13.962317 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b821cb5a-c26c-41a1-93ce-869e0e026410 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:13.996938 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb837a0f-0e5b-47e1-807f-579a06b5c418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:13.997464 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb837a0f-0e5b-47e1-807f-579a06b5c418 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:13.997738 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb837a0f-0e5b-47e1-807f-579a06b5c418 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:13.998000 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb837a0f-0e5b-47e1-807f-579a06b5c418 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:13.998675 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb837a0f-0e5b-47e1-807f-579a06b5c418 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:25:13.998976 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb837a0f-0e5b-47e1-807f-579a06b5c418 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:14.000387 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb837a0f-0e5b-47e1-807f-579a06b5c418 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:14.017445 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 394/800] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Oct 27 09:25:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:14.019064 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e8f66b-052a-4613-9a7a-285ed6e16f57 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:14.123259 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e8f66b-052a-4613-9a7a-285ed6e16f57 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:14.155456 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5745bdaa-26f0-460c-bcd2-e7232fd82983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:14.156098 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745bdaa-26f0-460c-bcd2-e7232fd82983 None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:14.156367 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745bdaa-26f0-460c-bcd2-e7232fd82983 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:14.156669 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745bdaa-26f0-460c-bcd2-e7232fd82983 None admin] PATH_INFO: `/v3/projects/16ac205ec6f241d69d5e6b8408f09f38/users/dbe4a6e4b6184826b30ac4fc13ad9b1f/roles/77d68d4713094fd59253e4a7d20e6f33` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:14.159018 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5745bdaa-26f0-460c-bcd2-e7232fd82983 None admin] RBAC: Authorizing `identity:create_grant(project_id=16ac205ec6f241d69d5e6b8408f09f38, user_id=dbe4a6e4b6184826b30ac4fc13ad9b1f, role_id=77d68d4713094fd59253e4a7d20e6f33)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:14.160449 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5745bdaa-26f0-460c-bcd2-e7232fd82983 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:14.174002 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 407/801] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Oct 27 09:25:14 2022] PUT /identity/v3/projects/16ac205ec6f241d69d5e6b8408f09f38/users/dbe4a6e4b6184826b30ac4fc13ad9b1f/roles/77d68d4713094fd59253e4a7d20e6f33 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:25:14.442435 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f1c91f-219a-4628-b22a-9c3830da237a None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:14.442829 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f1c91f-219a-4628-b22a-9c3830da237a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:14.443024 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f1c91f-219a-4628-b22a-9c3830da237a None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:14.444447 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 395/802] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:25:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:25:14.449639 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba9f04-74ae-4f17-90e2-8deed34a8898 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:14.449932 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba9f04-74ae-4f17-90e2-8deed34a8898 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:14.450210 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba9f04-74ae-4f17-90e2-8deed34a8898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:14.469145 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-46ba9f04-74ae-4f17-90e2-8deed34a8898 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:25:14.470330 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-46ba9f04-74ae-4f17-90e2-8deed34a8898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:25:14.512931 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:14.543691 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 408/803] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:25:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 09:25:14.812687 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f1c91f-219a-4628-b22a-9c3830da237a None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:14.917661 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f1c91f-219a-4628-b22a-9c3830da237a None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:14.947473 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:14.956565 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528e2302-f7e7-4eeb-a4db-515fe16259e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:14.956565 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e2302-f7e7-4eeb-a4db-515fe16259e0 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:14.956565 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e2302-f7e7-4eeb-a4db-515fe16259e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:14.956565 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e2302-f7e7-4eeb-a4db-515fe16259e0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:14.956933 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528e2302-f7e7-4eeb-a4db-515fe16259e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:14.957179 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528e2302-f7e7-4eeb-a4db-515fe16259e0 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:14.959009 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 396/804] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:25:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:14.968408 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ba9f04-74ae-4f17-90e2-8deed34a8898 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:15.068487 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ba9f04-74ae-4f17-90e2-8deed34a8898 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:15.102461 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d35d4e6-4970-4d5e-a9fa-89e682b7325f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:15.103061 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d35d4e6-4970-4d5e-a9fa-89e682b7325f None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:15.103339 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d35d4e6-4970-4d5e-a9fa-89e682b7325f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:15.103608 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d35d4e6-4970-4d5e-a9fa-89e682b7325f None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:15.104260 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d35d4e6-4970-4d5e-a9fa-89e682b7325f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:15.105730 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d35d4e6-4970-4d5e-a9fa-89e682b7325f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:15.128618 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 409/805] 10.222.0.70 () {60 vars in 1132 bytes} [Thu Oct 27 09:25:14 2022] POST /identity/v3/users => generated 285 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:25:15.916676 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a0e46-2ee8-4f72-9134-10d5cb3fd92b None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:15.917225 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a0e46-2ee8-4f72-9134-10d5cb3fd92b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:15.917225 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a0e46-2ee8-4f72-9134-10d5cb3fd92b None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:15.918659 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 397/806] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:25:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:25:15.923611 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3461b029-2094-45bd-a70f-aa9af04637f3 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:15.923897 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3461b029-2094-45bd-a70f-aa9af04637f3 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:15.924167 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3461b029-2094-45bd-a70f-aa9af04637f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:15.941934 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-3461b029-2094-45bd-a70f-aa9af04637f3 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:25:15.943151 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3461b029-2094-45bd-a70f-aa9af04637f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:25:15.969372 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:16.016838 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 410/807] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:25:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 09:25:16.282120 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a0e46-2ee8-4f72-9134-10d5cb3fd92b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:16.337514 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8172c2-bc48-43fe-8179-58f530fef104 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:16.337810 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8172c2-bc48-43fe-8179-58f530fef104 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:16.338087 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8172c2-bc48-43fe-8179-58f530fef104 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:16.339423 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 411/808] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 27 09:25:16.344785 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c2e03c-2dfe-408c-887d-eb419d801ab4 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:16.345139 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c2e03c-2dfe-408c-887d-eb419d801ab4 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:16.345483 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c2e03c-2dfe-408c-887d-eb419d801ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:16.368544 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-84c2e03c-2dfe-408c-887d-eb419d801ab4 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:25:16.368544 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-84c2e03c-2dfe-408c-887d-eb419d801ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:25:16.387843 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a0e46-2ee8-4f72-9134-10d5cb3fd92b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:16.422917 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6592a5-7a3b-4833-af9c-b3a23c12e37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:16.423522 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6592a5-7a3b-4833-af9c-b3a23c12e37b None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:16.423799 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6592a5-7a3b-4833-af9c-b3a23c12e37b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:16.424070 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6592a5-7a3b-4833-af9c-b3a23c12e37b None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:16.424755 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6592a5-7a3b-4833-af9c-b3a23c12e37b None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:16.426123 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6592a5-7a3b-4833-af9c-b3a23c12e37b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:16.433850 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.sql.core [None req-5b6592a5-7a3b-4833-af9c-b3a23c12e37b None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 27 09:25:16.433850 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 27 09:25:16.433850 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [parameters: {'id': '6c0c51f53aac4173b87e630b226a633e', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 27 09:25:16.433850 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81437) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 27 09:25:16.435160 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 412/809] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:25:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 09:25:16.440966 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-5b6592a5-7a3b-4833-af9c-b3a23c12e37b None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 27 09:25:16.440966 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:16.440966 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 27 09:25:16.440966 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 27 09:25:16.440966 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 27 09:25:16.440966 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 27 09:25:16.440966 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 27 09:25:16.440966 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application result = self._query(query) Oct 27 09:25:16.440966 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 27 09:25:16.440966 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application conn.query(q) Oct 27 09:25:16.440966 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 27 09:25:16.440966 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 27 09:25:16.440966 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 27 09:25:16.440966 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application result.read() Oct 27 09:25:16.441778 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 27 09:25:16.441778 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 27 09:25:16.441778 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 27 09:25:16.441778 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 27 09:25:16.441778 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 27 09:25:16.441778 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 27 09:25:16.441778 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 27 09:25:16.441778 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 27 09:25:16.441778 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 27 09:25:16.441778 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:25:16.441778 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 27 09:25:16.441778 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:25:16.441778 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:16.441778 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 27 09:25:16.441778 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 27 09:25:16.441778 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 27 09:25:16.441778 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 27 09:25:16.441778 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 27 09:25:16.443247 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application next(self.gen) Oct 27 09:25:16.443247 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 27 09:25:16.443247 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application yield resource Oct 27 09:25:16.443247 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 27 09:25:16.443247 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application next(self.gen) Oct 27 09:25:16.443247 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 27 09:25:16.443247 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self.session.rollback() Oct 27 09:25:16.443247 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 27 09:25:16.443247 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self.force_reraise() Oct 27 09:25:16.443247 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 27 09:25:16.443247 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise self.value Oct 27 09:25:16.443247 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 27 09:25:16.443247 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 27 09:25:16.443247 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 27 09:25:16.443247 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application session.commit() Oct 27 09:25:16.443247 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 27 09:25:16.443247 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 27 09:25:16.444113 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 27 09:25:16.444113 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._prepare_impl() Oct 27 09:25:16.444113 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 27 09:25:16.444113 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self.session.flush() Oct 27 09:25:16.444113 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 27 09:25:16.444113 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._flush(objects) Oct 27 09:25:16.444113 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 27 09:25:16.444113 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 27 09:25:16.444113 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 27 09:25:16.444113 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application compat.raise_( Oct 27 09:25:16.444113 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 27 09:25:16.444113 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception Oct 27 09:25:16.444113 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 27 09:25:16.444113 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application flush_context.execute() Oct 27 09:25:16.444113 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 27 09:25:16.444113 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rec.execute(self) Oct 27 09:25:16.444113 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 27 09:25:16.446441 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 27 09:25:16.446441 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 27 09:25:16.446441 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 27 09:25:16.446441 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 27 09:25:16.446441 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 27 09:25:16.446441 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 27 09:25:16.446441 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 27 09:25:16.446441 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 27 09:25:16.446441 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 27 09:25:16.446441 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 27 09:25:16.446441 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 27 09:25:16.446441 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 27 09:25:16.446441 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 27 09:25:16.446441 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 27 09:25:16.446441 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 27 09:25:16.447912 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 27 09:25:16.447912 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception Oct 27 09:25:16.447912 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 27 09:25:16.447912 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 27 09:25:16.447912 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 27 09:25:16.447912 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 27 09:25:16.447912 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 27 09:25:16.447912 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application result = self._query(query) Oct 27 09:25:16.447912 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 27 09:25:16.447912 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application conn.query(q) Oct 27 09:25:16.447912 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 27 09:25:16.447912 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 27 09:25:16.447912 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 27 09:25:16.447912 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application result.read() Oct 27 09:25:16.447912 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 27 09:25:16.447912 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 27 09:25:16.447912 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 27 09:25:16.449181 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 27 09:25:16.449181 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 27 09:25:16.449181 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 27 09:25:16.449181 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 27 09:25:16.449181 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 27 09:25:16.449181 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 27 09:25:16.449181 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 27 09:25:16.449181 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [parameters: {'id': '6c0c51f53aac4173b87e630b226a633e', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 27 09:25:16.449181 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 27 09:25:16.449181 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:25:16.449181 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 09:25:16.449181 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:25:16.449181 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:16.449181 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 27 09:25:16.449181 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 27 09:25:16.450142 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 27 09:25:16.450142 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 27 09:25:16.450142 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 27 09:25:16.450142 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:25:16.450142 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 09:25:16.450142 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:25:16.450142 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:16.450142 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:25:16.450142 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:25:16.450142 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:25:16.450142 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:25:16.450142 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:25:16.450142 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:25:16.450142 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:25:16.450142 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:25:16.450142 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:25:16.450142 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:25:16.451231 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 27 09:25:16.451231 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 27 09:25:16.451231 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:25:16.451231 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:25:16.451231 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 27 09:25:16.451231 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 27 09:25:16.451231 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:25:16.451231 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:25:16.451231 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 27 09:25:16.451231 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 27 09:25:16.451231 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 27 09:25:16.451231 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:25:16.451231 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 398/810] 10.222.0.70 () {60 vars in 1137 bytes} [Thu Oct 27 09:25:16 2022] POST /identity/v3/domains => generated 190 bytes in 162 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 27 09:25:16.452058 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c2e03c-2dfe-408c-887d-eb419d801ab4 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:16.542048 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:16.551522 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c2e03c-2dfe-408c-887d-eb419d801ab4 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:16.587581 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8a48a1-56ba-43d2-a1ea-98af6d3800dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:16.588315 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8a48a1-56ba-43d2-a1ea-98af6d3800dc None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:16.588730 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8a48a1-56ba-43d2-a1ea-98af6d3800dc None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:16.589093 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8a48a1-56ba-43d2-a1ea-98af6d3800dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:16.599247 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8a48a1-56ba-43d2-a1ea-98af6d3800dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:16.600779 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8a48a1-56ba-43d2-a1ea-98af6d3800dc None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:16.615626 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-dc8a48a1-56ba-43d2-a1ea-98af6d3800dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:25:16.615626 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:16.615626 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:25:16.615626 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:25:16.615626 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:25:16.615626 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:25:16.615626 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:25:16.615626 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:25:16.615626 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:25:16.615626 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:25:16.615626 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:25:16.615626 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:25:16.615626 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:25:16.615626 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:25:16.615626 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:25:16.615626 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:25:16.616740 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:25:16.616740 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:25:16.616740 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:25:16.616740 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:25:16.616740 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:25:16.616740 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:25:16.616740 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:25:16.616740 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:25:16.616740 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:25:16.616740 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:25:16.616740 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:25:16.616740 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:25:16.616740 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:25:16.616740 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:25:16.616740 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:25:16.616740 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:25:16.616740 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:25:16.617836 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:25:16.617836 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:25:16.617836 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:25:16.620663 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 413/811] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:25:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:25:16.624775 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6592a5-7a3b-4833-af9c-b3a23c12e37b None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:16.694567 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:16.709777 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8a48a1-56ba-43d2-a1ea-98af6d3800dc None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:16.738021 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6592a5-7a3b-4833-af9c-b3a23c12e37b None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:16.776894 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ed7833-cce6-4aa7-824d-340a2a154b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:16.777497 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed7833-cce6-4aa7-824d-340a2a154b4a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:16.777774 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed7833-cce6-4aa7-824d-340a2a154b4a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:16.778044 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed7833-cce6-4aa7-824d-340a2a154b4a None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:16.778678 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ed7833-cce6-4aa7-824d-340a2a154b4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:25:16.778978 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ed7833-cce6-4aa7-824d-340a2a154b4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:16.780341 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ed7833-cce6-4aa7-824d-340a2a154b4a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:16.800650 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 399/812] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:25:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:16.829916 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8a48a1-56ba-43d2-a1ea-98af6d3800dc None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:16.868818 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c5804-0ddd-49dc-8ca6-6f607076ed9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:16.869561 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c5804-0ddd-49dc-8ca6-6f607076ed9c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:16.869909 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c5804-0ddd-49dc-8ca6-6f607076ed9c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:16.870248 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c5804-0ddd-49dc-8ca6-6f607076ed9c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:16.882456 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3c5804-0ddd-49dc-8ca6-6f607076ed9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:25:16.887583 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-2a3c5804-0ddd-49dc-8ca6-6f607076ed9c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 27 09:25:16.887583 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:16.887583 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:25:16.887583 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:25:16.887583 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:25:16.887583 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:25:16.887583 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:25:16.887583 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:25:16.887583 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:25:16.887583 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:25:16.887583 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:25:16.887583 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:25:16.887583 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:25:16.887583 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:25:16.887583 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:25:16.887583 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:25:16.888910 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:25:16.888910 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:25:16.888910 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:25:16.888910 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:25:16.888910 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:25:16.888910 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:25:16.888910 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:25:16.888910 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:25:16.888910 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:25:16.888910 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:25:16.888910 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:25:16.888910 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:25:16.888910 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:25:16.888910 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:25:16.888910 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:25:16.888910 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:25:16.889980 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:25:16.889980 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:25:16.889980 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:25:16.889980 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:25:16.889980 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:25:16.889980 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:25:16.889980 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:25:16.889980 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:25:16.889980 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:25:16.889980 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:25:16.889980 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 27 09:25:16.889980 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:25:16.893085 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 414/813] 10.222.0.70 () {56 vars in 1120 bytes} [Thu Oct 27 09:25:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 09:25:16.897289 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ed7833-cce6-4aa7-824d-340a2a154b4a None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:17.011178 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ed7833-cce6-4aa7-824d-340a2a154b4a None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:17.036868 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:17.052448 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb809bb9-885c-4c78-b2a4-3f9f7fb68277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:17.053063 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb809bb9-885c-4c78-b2a4-3f9f7fb68277 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:17.053350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb809bb9-885c-4c78-b2a4-3f9f7fb68277 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:17.053617 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb809bb9-885c-4c78-b2a4-3f9f7fb68277 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:17.054274 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb809bb9-885c-4c78-b2a4-3f9f7fb68277 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:25:17.054578 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb809bb9-885c-4c78-b2a4-3f9f7fb68277 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:17.055940 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb809bb9-885c-4c78-b2a4-3f9f7fb68277 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:17.069697 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 400/814] 10.222.0.70 () {56 vars in 1115 bytes} [Thu Oct 27 09:25:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:17.071020 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c5804-0ddd-49dc-8ca6-6f607076ed9c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:17.076406 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:17.175453 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c5804-0ddd-49dc-8ca6-6f607076ed9c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:17.208002 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41ab4cf-2c69-4a38-a196-2e7a9724bc3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:17.208653 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ab4cf-2c69-4a38-a196-2e7a9724bc3d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:17.208934 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ab4cf-2c69-4a38-a196-2e7a9724bc3d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:17.209204 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ab4cf-2c69-4a38-a196-2e7a9724bc3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:17.220225 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41ab4cf-2c69-4a38-a196-2e7a9724bc3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:17.221735 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41ab4cf-2c69-4a38-a196-2e7a9724bc3d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:17.237803 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-b41ab4cf-2c69-4a38-a196-2e7a9724bc3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:25:17.237803 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:17.237803 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:25:17.237803 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:25:17.237803 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:25:17.237803 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:25:17.237803 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:25:17.237803 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:25:17.237803 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:25:17.237803 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:25:17.237803 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:25:17.237803 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:25:17.237803 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:25:17.237803 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:25:17.237803 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:25:17.237803 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:25:17.239974 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:25:17.239974 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:25:17.239974 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:25:17.239974 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:25:17.239974 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:25:17.239974 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:25:17.239974 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:25:17.239974 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:25:17.239974 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:25:17.239974 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:25:17.239974 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:25:17.239974 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:25:17.239974 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:25:17.239974 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:25:17.239974 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:25:17.239974 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:25:17.239974 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:25:17.241405 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:25:17.241405 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:25:17.241405 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:25:17.241405 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 415/815] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:25:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:25:17.245581 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb809bb9-885c-4c78-b2a4-3f9f7fb68277 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:17.341799 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b293e-3187-4043-baa5-4bea5a864b54 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:17.342090 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b293e-3187-4043-baa5-4bea5a864b54 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:17.342373 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b293e-3187-4043-baa5-4bea5a864b54 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:17.343975 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 416/816] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:25:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 27 09:25:17.346242 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb809bb9-885c-4c78-b2a4-3f9f7fb68277 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:17.349628 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef7947-20fa-489f-955f-ca4086cece4f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:17.349910 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef7947-20fa-489f-955f-ca4086cece4f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:17.350183 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef7947-20fa-489f-955f-ca4086cece4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:17.367692 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-aeef7947-20fa-489f-955f-ca4086cece4f None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:25:17.368906 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-aeef7947-20fa-489f-955f-ca4086cece4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:25:17.380056 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c26c944-c7ba-4c00-b4e6-fcad1bb240a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:17.380684 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26c944-c7ba-4c00-b4e6-fcad1bb240a2 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:17.380960 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26c944-c7ba-4c00-b4e6-fcad1bb240a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:17.381227 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26c944-c7ba-4c00-b4e6-fcad1bb240a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:17.381860 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c26c944-c7ba-4c00-b4e6-fcad1bb240a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:25:17.382158 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c26c944-c7ba-4c00-b4e6-fcad1bb240a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:17.383509 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c26c944-c7ba-4c00-b4e6-fcad1bb240a2 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:17.395709 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 401/817] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:25:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:17.400975 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c26c944-c7ba-4c00-b4e6-fcad1bb240a2 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:17.420083 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:17.436556 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 417/818] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:25:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 27 09:25:17.447076 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:17.498941 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c26c944-c7ba-4c00-b4e6-fcad1bb240a2 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:17.531363 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b05c315-38c4-439f-8352-a880252c3dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:17.531975 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05c315-38c4-439f-8352-a880252c3dcd None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:17.532249 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05c315-38c4-439f-8352-a880252c3dcd None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:17.532541 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05c315-38c4-439f-8352-a880252c3dcd None admin] PATH_INFO: `/v3/users/nova` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:17.539849 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b05c315-38c4-439f-8352-a880252c3dcd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:17.541299 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b05c315-38c4-439f-8352-a880252c3dcd None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:17.552450 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-0b05c315-38c4-439f-8352-a880252c3dcd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 27 09:25:17.552450 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:17.552450 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:25:17.552450 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:25:17.552450 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:25:17.552450 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:25:17.552450 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:25:17.552450 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:25:17.552450 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:25:17.552450 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:25:17.552450 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:25:17.552450 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:25:17.552450 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 09:25:17.552450 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 09:25:17.552450 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 09:25:17.552450 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 09:25:17.553554 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:25:17.553554 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:25:17.553554 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 09:25:17.553554 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:25:17.553554 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 09:25:17.553554 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:25:17.553554 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:25:17.553554 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:25:17.553554 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:25:17.553554 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:25:17.553554 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:25:17.553554 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:25:17.553554 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:25:17.553554 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:25:17.553554 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:25:17.553554 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:25:17.553554 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:25:17.553554 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:25:17.554847 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:25:17.554847 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:25:17.554847 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 09:25:17.554847 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 09:25:17.554847 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 09:25:17.554847 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 09:25:17.554847 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 09:25:17.554847 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 09:25:17.554847 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 27 09:25:17.554847 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:25:17.554847 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 402/819] 10.222.0.70 () {56 vars in 1093 bytes} [Thu Oct 27 09:25:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 09:25:17.559191 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef7947-20fa-489f-955f-ca4086cece4f None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:17.655096 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef7947-20fa-489f-955f-ca4086cece4f None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:17.688167 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393b1d78-b2da-449f-a006-c6b58075ba0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:17.688783 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b1d78-b2da-449f-a006-c6b58075ba0f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:17.689062 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b1d78-b2da-449f-a006-c6b58075ba0f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:17.689332 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b1d78-b2da-449f-a006-c6b58075ba0f None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:17.690047 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393b1d78-b2da-449f-a006-c6b58075ba0f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:25:17.690347 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393b1d78-b2da-449f-a006-c6b58075ba0f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:17.691773 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393b1d78-b2da-449f-a006-c6b58075ba0f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:17.705223 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b05c315-38c4-439f-8352-a880252c3dcd None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:17.709841 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 418/820] 10.222.0.70 () {56 vars in 1133 bytes} [Thu Oct 27 09:25:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:17.715186 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393b1d78-b2da-449f-a006-c6b58075ba0f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:17.761549 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:17.805969 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b05c315-38c4-439f-8352-a880252c3dcd None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:17.812816 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:17.816129 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393b1d78-b2da-449f-a006-c6b58075ba0f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:17.841587 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477f2dfa-58ba-43fd-84c0-b03335586c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:17.842200 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477f2dfa-58ba-43fd-84c0-b03335586c12 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:17.842470 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477f2dfa-58ba-43fd-84c0-b03335586c12 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:17.842738 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477f2dfa-58ba-43fd-84c0-b03335586c12 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:17.851120 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae372e60-4133-44e3-93b8-6009285dbea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:17.851705 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae372e60-4133-44e3-93b8-6009285dbea3 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:17.851976 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae372e60-4133-44e3-93b8-6009285dbea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:17.852237 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae372e60-4133-44e3-93b8-6009285dbea3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:17.852946 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477f2dfa-58ba-43fd-84c0-b03335586c12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:25:17.857668 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-477f2dfa-58ba-43fd-84c0-b03335586c12 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 09:25:17.857668 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:17.857668 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:25:17.857668 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:25:17.857668 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:25:17.857668 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:25:17.857668 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:25:17.857668 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:25:17.857668 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:25:17.857668 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:25:17.857668 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:25:17.857668 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:25:17.857668 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:25:17.857668 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:25:17.857668 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:25:17.857668 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:25:17.858754 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:25:17.858754 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:25:17.858754 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:25:17.858754 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:25:17.858754 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:25:17.858754 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:25:17.858754 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:25:17.858754 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:25:17.858754 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:25:17.858754 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:25:17.858754 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:25:17.858754 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:25:17.858754 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:25:17.858754 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:25:17.858754 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:25:17.858754 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:25:17.859792 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:25:17.859792 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:25:17.859792 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:25:17.859792 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:25:17.859792 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:25:17.859792 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:25:17.859792 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:25:17.859792 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:25:17.859792 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:25:17.859792 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:25:17.859792 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 09:25:17.859792 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:25:17.859792 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 403/821] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Oct 27 09:25:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 09:25:17.864922 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477f2dfa-58ba-43fd-84c0-b03335586c12 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:17.970707 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477f2dfa-58ba-43fd-84c0-b03335586c12 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:18.004254 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ccd20f-d8b8-4f79-806f-050af5d4ccf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:18.005688 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ccd20f-d8b8-4f79-806f-050af5d4ccf3 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:18.005964 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ccd20f-d8b8-4f79-806f-050af5d4ccf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:18.006227 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ccd20f-d8b8-4f79-806f-050af5d4ccf3 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:18.006871 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ccd20f-d8b8-4f79-806f-050af5d4ccf3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:25:18.007166 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ccd20f-d8b8-4f79-806f-050af5d4ccf3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:18.008534 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ccd20f-d8b8-4f79-806f-050af5d4ccf3 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:18.016340 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 404/822] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Oct 27 09:25:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:25:18.022883 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ccd20f-d8b8-4f79-806f-050af5d4ccf3 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:18.033897 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae372e60-4133-44e3-93b8-6009285dbea3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:18.035339 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae372e60-4133-44e3-93b8-6009285dbea3 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:18.049994 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:18.079329 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:18.121690 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ccd20f-d8b8-4f79-806f-050af5d4ccf3 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:18.135712 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 419/823] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Oct 27 09:25:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 27 09:25:18.141199 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae372e60-4133-44e3-93b8-6009285dbea3 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:18.155401 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd86322-07af-4ffa-921e-351a54b058a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:18.156022 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd86322-07af-4ffa-921e-351a54b058a0 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:18.156296 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd86322-07af-4ffa-921e-351a54b058a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:18.156587 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd86322-07af-4ffa-921e-351a54b058a0 None admin] PATH_INFO: `/v3/users/c46572e4f2b94e45ae058178a2245f6a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:18.169374 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd86322-07af-4ffa-921e-351a54b058a0 None admin] RBAC: Authorizing `identity:get_user(user_id=c46572e4f2b94e45ae058178a2245f6a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:18.170790 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd86322-07af-4ffa-921e-351a54b058a0 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:18.172359 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 405/824] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Oct 27 09:25:18 2022] GET /identity/v3/users/c46572e4f2b94e45ae058178a2245f6a => generated 285 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:25:18.178442 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd86322-07af-4ffa-921e-351a54b058a0 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:18.235764 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae372e60-4133-44e3-93b8-6009285dbea3 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:18.260875 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:18.272673 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225168dd-d20d-44e8-bccd-e46576c4e2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:18.273299 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225168dd-d20d-44e8-bccd-e46576c4e2d7 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:18.273567 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225168dd-d20d-44e8-bccd-e46576c4e2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:18.273839 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225168dd-d20d-44e8-bccd-e46576c4e2d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:18.277434 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd86322-07af-4ffa-921e-351a54b058a0 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:18.284276 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225168dd-d20d-44e8-bccd-e46576c4e2d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:18.285205 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225168dd-d20d-44e8-bccd-e46576c4e2d7 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:18.287721 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:18.300132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-225168dd-d20d-44e8-bccd-e46576c4e2d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:25:18.300132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:18.300132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:25:18.300132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:25:18.300132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:25:18.300132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:25:18.300132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:25:18.300132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:25:18.300132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:25:18.300132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:25:18.300132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:25:18.300132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:25:18.300132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:25:18.300132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:25:18.300132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:25:18.300132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:25:18.301205 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:25:18.301205 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:25:18.301205 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:25:18.301205 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:25:18.301205 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:25:18.301205 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:25:18.301205 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:25:18.301205 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:25:18.301205 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:25:18.301205 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:25:18.301205 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:25:18.301205 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:25:18.301205 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:25:18.301205 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:25:18.301205 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:25:18.301205 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:25:18.301205 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:25:18.302635 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:25:18.302635 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:25:18.302635 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:25:18.302635 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 420/825] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:25:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 09:25:18.307351 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225168dd-d20d-44e8-bccd-e46576c4e2d7 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:18.318399 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed3e424-492b-4071-bd03-c897a512fe3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:18.319021 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed3e424-492b-4071-bd03-c897a512fe3e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:18.319294 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed3e424-492b-4071-bd03-c897a512fe3e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:18.319565 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed3e424-492b-4071-bd03-c897a512fe3e None admin] PATH_INFO: `/v3/projects/357196595fb544bdbe85994308761315` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:18.320694 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed3e424-492b-4071-bd03-c897a512fe3e None admin] RBAC: Authorizing `identity:get_project(project_id=357196595fb544bdbe85994308761315)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:18.322110 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed3e424-492b-4071-bd03-c897a512fe3e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:18.323717 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 406/826] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Oct 27 09:25:18 2022] GET /identity/v3/projects/357196595fb544bdbe85994308761315 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:25:18.329461 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed3e424-492b-4071-bd03-c897a512fe3e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:18.404193 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225168dd-d20d-44e8-bccd-e46576c4e2d7 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:18.427350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed3e424-492b-4071-bd03-c897a512fe3e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:18.438242 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138dc945-74b0-421b-8409-b31525199dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:18.438840 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138dc945-74b0-421b-8409-b31525199dc7 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:18.439122 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138dc945-74b0-421b-8409-b31525199dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:18.439392 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138dc945-74b0-421b-8409-b31525199dc7 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:18.440011 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138dc945-74b0-421b-8409-b31525199dc7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:25:18.440313 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138dc945-74b0-421b-8409-b31525199dc7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:18.441716 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138dc945-74b0-421b-8409-b31525199dc7 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:18.453546 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 421/827] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:25:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:25:18.458734 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138dc945-74b0-421b-8409-b31525199dc7 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:18.459818 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c2b9a8-6209-4fd1-bb20-e6e47e81ea0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:18.460459 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c2b9a8-6209-4fd1-bb20-e6e47e81ea0d None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:18.460771 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c2b9a8-6209-4fd1-bb20-e6e47e81ea0d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:18.461041 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c2b9a8-6209-4fd1-bb20-e6e47e81ea0d None admin] PATH_INFO: `/v3/projects/357196595fb544bdbe85994308761315/users/c46572e4f2b94e45ae058178a2245f6a/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:18.462918 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c2b9a8-6209-4fd1-bb20-e6e47e81ea0d None admin] RBAC: Authorizing `identity:create_grant(project_id=357196595fb544bdbe85994308761315, user_id=c46572e4f2b94e45ae058178a2245f6a, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:18.464324 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c2b9a8-6209-4fd1-bb20-e6e47e81ea0d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:18.478065 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 407/828] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Oct 27 09:25:18 2022] PUT /identity/v3/projects/357196595fb544bdbe85994308761315/users/c46572e4f2b94e45ae058178a2245f6a/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:25:18.516482 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:18.563157 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138dc945-74b0-421b-8409-b31525199dc7 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:18.599588 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99325dd2-4809-4adc-a25e-bdb58c489150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:18.600239 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99325dd2-4809-4adc-a25e-bdb58c489150 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:18.600538 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99325dd2-4809-4adc-a25e-bdb58c489150 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:18.600813 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99325dd2-4809-4adc-a25e-bdb58c489150 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:18.608198 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99325dd2-4809-4adc-a25e-bdb58c489150 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:18.609671 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99325dd2-4809-4adc-a25e-bdb58c489150 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:18.623065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-99325dd2-4809-4adc-a25e-bdb58c489150 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:25:18.623065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:18.623065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:25:18.623065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:25:18.623065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:25:18.623065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:25:18.623065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:25:18.623065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:25:18.623065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:25:18.623065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:25:18.623065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:25:18.623065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:25:18.623065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 09:25:18.623065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 09:25:18.623065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 09:25:18.624162 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 09:25:18.624162 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:25:18.624162 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:25:18.624162 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:25:18.624162 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:25:18.624162 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:25:18.624162 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:25:18.624162 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:25:18.624162 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:25:18.624162 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:25:18.624162 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:25:18.624162 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:25:18.624162 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:25:18.624162 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:25:18.624162 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:25:18.624162 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:25:18.624162 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:25:18.625110 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 09:25:18.625110 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 09:25:18.625110 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 09:25:18.625110 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 09:25:18.625110 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 09:25:18.625110 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 09:25:18.625110 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:25:18.625110 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:25:18.625110 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 422/829] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Oct 27 09:25:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 09:25:18.630258 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c2b9a8-6209-4fd1-bb20-e6e47e81ea0d None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:18.646947 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:18.732778 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c2b9a8-6209-4fd1-bb20-e6e47e81ea0d None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:18.766203 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f98a4f-2efe-4ee2-adc3-db2113e87236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:18.766850 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f98a4f-2efe-4ee2-adc3-db2113e87236 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:18.767071 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f98a4f-2efe-4ee2-adc3-db2113e87236 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:18.767335 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f98a4f-2efe-4ee2-adc3-db2113e87236 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:18.768009 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f98a4f-2efe-4ee2-adc3-db2113e87236 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:25:18.768308 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f98a4f-2efe-4ee2-adc3-db2113e87236 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:18.769769 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f98a4f-2efe-4ee2-adc3-db2113e87236 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:18.781530 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 408/830] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Oct 27 09:25:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:18.788212 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99325dd2-4809-4adc-a25e-bdb58c489150 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:18.884814 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99325dd2-4809-4adc-a25e-bdb58c489150 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:18.900349 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:18.919222 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e82cac-1270-4ca1-95cd-5b3054b9dc2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:18.919822 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e82cac-1270-4ca1-95cd-5b3054b9dc2c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:18.920097 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e82cac-1270-4ca1-95cd-5b3054b9dc2c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:18.920368 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e82cac-1270-4ca1-95cd-5b3054b9dc2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:18.921110 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e82cac-1270-4ca1-95cd-5b3054b9dc2c None admin] RBAC: Adding query filter params (role.id=77d68d4713094fd59253e4a7d20e6f33, scope.project.id=16ac205ec6f241d69d5e6b8408f09f38, user.id=dbe4a6e4b6184826b30ac4fc13ad9b1f) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:25:18.921417 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e82cac-1270-4ca1-95cd-5b3054b9dc2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:18.922836 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e82cac-1270-4ca1-95cd-5b3054b9dc2c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:18.929122 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 423/831] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Oct 27 09:25:18 2022] GET /identity/v3/role_assignments?user.id=dbe4a6e4b6184826b30ac4fc13ad9b1f&scope.project.id=16ac205ec6f241d69d5e6b8408f09f38&role.id=77d68d4713094fd59253e4a7d20e6f33 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:20.596467 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890ca77-8488-48a2-bebf-2695e7915897 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:20.597012 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890ca77-8488-48a2-bebf-2695e7915897 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:20.597012 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890ca77-8488-48a2-bebf-2695e7915897 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:20.598972 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 409/832] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:25:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:25:20.604268 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d223cacc-c2ef-4a16-b0ae-0bb00e24fd64 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:20.604694 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d223cacc-c2ef-4a16-b0ae-0bb00e24fd64 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:20.605604 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d223cacc-c2ef-4a16-b0ae-0bb00e24fd64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:20.623372 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-d223cacc-c2ef-4a16-b0ae-0bb00e24fd64 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:25:20.624590 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d223cacc-c2ef-4a16-b0ae-0bb00e24fd64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:25:20.688695 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 424/833] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:25:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 09:25:20.957918 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f890ca77-8488-48a2-bebf-2695e7915897 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:21.034056 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:21.059253 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f890ca77-8488-48a2-bebf-2695e7915897 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:21.092900 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4d158e-551b-44d6-a273-d6e3af514bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:21.093762 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d158e-551b-44d6-a273-d6e3af514bf9 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:21.094046 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d158e-551b-44d6-a273-d6e3af514bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:21.094311 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d158e-551b-44d6-a273-d6e3af514bf9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:21.104161 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4d158e-551b-44d6-a273-d6e3af514bf9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:25:21.108921 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-ee4d158e-551b-44d6-a273-d6e3af514bf9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 09:25:21.108921 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:21.108921 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:25:21.108921 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:25:21.108921 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:25:21.108921 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:25:21.108921 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:25:21.108921 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:25:21.108921 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:25:21.108921 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:25:21.108921 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:25:21.108921 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:25:21.108921 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:25:21.108921 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:25:21.108921 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:25:21.108921 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:25:21.110010 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:25:21.110010 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:25:21.110010 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:25:21.110010 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:25:21.110010 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:25:21.110010 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:25:21.110010 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:25:21.110010 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:25:21.110010 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:25:21.110010 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:25:21.110010 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:25:21.110010 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:25:21.110010 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:25:21.110010 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:25:21.110010 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:25:21.110010 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:25:21.111307 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:25:21.111307 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:25:21.111307 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:25:21.111307 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:25:21.111307 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:25:21.111307 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:25:21.111307 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:25:21.111307 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:25:21.111307 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:25:21.111307 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:25:21.111307 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 09:25:21.111307 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:25:21.111307 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 410/834] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Oct 27 09:25:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:25:21.116764 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d223cacc-c2ef-4a16-b0ae-0bb00e24fd64 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:21.213868 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d223cacc-c2ef-4a16-b0ae-0bb00e24fd64 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:21.224299 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:21.247308 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379a7aad-6542-4604-bed8-fa9e12f7cbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:21.247907 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a7aad-6542-4604-bed8-fa9e12f7cbf2 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:21.248180 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a7aad-6542-4604-bed8-fa9e12f7cbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:21.248446 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a7aad-6542-4604-bed8-fa9e12f7cbf2 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:21.249124 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379a7aad-6542-4604-bed8-fa9e12f7cbf2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:25:21.249420 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379a7aad-6542-4604-bed8-fa9e12f7cbf2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:21.250764 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379a7aad-6542-4604-bed8-fa9e12f7cbf2 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:21.259147 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 425/835] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Oct 27 09:25:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:21.265058 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4d158e-551b-44d6-a273-d6e3af514bf9 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:21.363234 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4d158e-551b-44d6-a273-d6e3af514bf9 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:21.396853 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a110feda-d6d3-4bcf-a838-55b717daa933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:21.397470 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110feda-d6d3-4bcf-a838-55b717daa933 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:21.397744 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110feda-d6d3-4bcf-a838-55b717daa933 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:21.398013 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110feda-d6d3-4bcf-a838-55b717daa933 None admin] PATH_INFO: `/v3/users/c46572e4f2b94e45ae058178a2245f6a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:21.399106 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a110feda-d6d3-4bcf-a838-55b717daa933 None admin] RBAC: Authorizing `identity:get_user(user_id=c46572e4f2b94e45ae058178a2245f6a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:21.400542 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a110feda-d6d3-4bcf-a838-55b717daa933 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:21.402106 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 411/836] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Oct 27 09:25:21 2022] GET /identity/v3/users/c46572e4f2b94e45ae058178a2245f6a => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:21.407982 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379a7aad-6542-4604-bed8-fa9e12f7cbf2 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:21.510742 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379a7aad-6542-4604-bed8-fa9e12f7cbf2 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:21.543016 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d5909-0aeb-4d33-ab8b-7e69bd9a49b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:21.543605 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d5909-0aeb-4d33-ab8b-7e69bd9a49b7 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:21.543882 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d5909-0aeb-4d33-ab8b-7e69bd9a49b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:21.544146 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d5909-0aeb-4d33-ab8b-7e69bd9a49b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:21.582747 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:21.608929 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8d5909-0aeb-4d33-ab8b-7e69bd9a49b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:21.610350 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8d5909-0aeb-4d33-ab8b-7e69bd9a49b7 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:21.703384 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 426/837] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Oct 27 09:25:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 27 09:25:21.709333 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a110feda-d6d3-4bcf-a838-55b717daa933 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:21.712186 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:21.806027 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a110feda-d6d3-4bcf-a838-55b717daa933 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:21.837807 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77604712-a885-40bb-84bb-0eeee170486f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:21.838414 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77604712-a885-40bb-84bb-0eeee170486f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:21.838687 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77604712-a885-40bb-84bb-0eeee170486f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:21.838951 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77604712-a885-40bb-84bb-0eeee170486f None admin] PATH_INFO: `/v3/projects/357196595fb544bdbe85994308761315` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:21.840031 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77604712-a885-40bb-84bb-0eeee170486f None admin] RBAC: Authorizing `identity:get_project(project_id=357196595fb544bdbe85994308761315)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:21.841458 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77604712-a885-40bb-84bb-0eeee170486f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:21.843039 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 412/838] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Oct 27 09:25:21 2022] GET /identity/v3/projects/357196595fb544bdbe85994308761315 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:21.850022 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d5909-0aeb-4d33-ab8b-7e69bd9a49b7 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:21.917537 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:21.946423 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d5909-0aeb-4d33-ab8b-7e69bd9a49b7 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:21.979483 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394221f3-4171-4145-828d-e3bc7da41a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:21.980067 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394221f3-4171-4145-828d-e3bc7da41a51 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:21.980338 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394221f3-4171-4145-828d-e3bc7da41a51 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:21.980631 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394221f3-4171-4145-828d-e3bc7da41a51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:21.981330 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394221f3-4171-4145-828d-e3bc7da41a51 None admin] RBAC: Adding query filter params (role.id=0dfd839b7a194aee96f82f4ebc1620a7, scope.project.id=357196595fb544bdbe85994308761315, user.id=c46572e4f2b94e45ae058178a2245f6a) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:25:21.981631 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394221f3-4171-4145-828d-e3bc7da41a51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:21.983026 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394221f3-4171-4145-828d-e3bc7da41a51 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:21.988512 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 427/839] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Oct 27 09:25:21 2022] GET /identity/v3/role_assignments?user.id=c46572e4f2b94e45ae058178a2245f6a&scope.project.id=357196595fb544bdbe85994308761315&role.id=0dfd839b7a194aee96f82f4ebc1620a7 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:24.122590 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52699e-9888-4454-8b4b-57a1a2400c4b None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:24.123119 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52699e-9888-4454-8b4b-57a1a2400c4b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:24.123119 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52699e-9888-4454-8b4b-57a1a2400c4b None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:24.124440 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 413/840] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:25:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:25:24.129557 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8be19-80d9-4146-af37-7206f37de9d8 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:24.129937 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8be19-80d9-4146-af37-7206f37de9d8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:24.130091 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8be19-80d9-4146-af37-7206f37de9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:24.147982 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-7ac8be19-80d9-4146-af37-7206f37de9d8 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:25:24.149169 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7ac8be19-80d9-4146-af37-7206f37de9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:25:24.211894 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 428/841] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:25:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 09:25:24.480784 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc52699e-9888-4454-8b4b-57a1a2400c4b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:24.548875 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:24.580171 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc52699e-9888-4454-8b4b-57a1a2400c4b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:24.613704 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc80d1ef-8ba9-420c-8d2c-24dd5d85af1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:24.614309 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80d1ef-8ba9-420c-8d2c-24dd5d85af1e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:24.614584 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80d1ef-8ba9-420c-8d2c-24dd5d85af1e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:24.614852 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80d1ef-8ba9-420c-8d2c-24dd5d85af1e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:24.615943 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc80d1ef-8ba9-420c-8d2c-24dd5d85af1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:24.617393 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc80d1ef-8ba9-420c-8d2c-24dd5d85af1e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:24.618916 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 414/842] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:25:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:24.624594 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac8be19-80d9-4146-af37-7206f37de9d8 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:24.687553 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:24.722100 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac8be19-80d9-4146-af37-7206f37de9d8 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:24.754705 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f7580-547d-4e9b-9b41-a3f8943490fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:24.755257 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f7580-547d-4e9b-9b41-a3f8943490fe None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:24.755529 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f7580-547d-4e9b-9b41-a3f8943490fe None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:24.755792 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f7580-547d-4e9b-9b41-a3f8943490fe None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:24.756424 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7f7580-547d-4e9b-9b41-a3f8943490fe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:24.757852 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7f7580-547d-4e9b-9b41-a3f8943490fe None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:24.778432 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 429/843] 10.222.0.70 () {60 vars in 1132 bytes} [Thu Oct 27 09:25:24 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:25:26.933608 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e745ada-5f55-4da4-a85d-66b9accc4b06 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:26.934139 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e745ada-5f55-4da4-a85d-66b9accc4b06 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:26.934139 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e745ada-5f55-4da4-a85d-66b9accc4b06 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:26.935465 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 415/844] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:25:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:25:26.940550 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec6957-abad-4177-81e3-89f3356ad6b6 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:26.940922 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec6957-abad-4177-81e3-89f3356ad6b6 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:26.941114 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec6957-abad-4177-81e3-89f3356ad6b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:26.959767 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-60ec6957-abad-4177-81e3-89f3356ad6b6 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:25:26.960987 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-60ec6957-abad-4177-81e3-89f3356ad6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:25:27.024424 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 430/845] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:25:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 09:25:27.292933 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e745ada-5f55-4da4-a85d-66b9accc4b06 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:27.391472 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e745ada-5f55-4da4-a85d-66b9accc4b06 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:27.423387 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47a2c02-5508-48ea-b9f0-afa3381d047d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:27.423998 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a2c02-5508-48ea-b9f0-afa3381d047d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:27.424273 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a2c02-5508-48ea-b9f0-afa3381d047d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:27.424558 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a2c02-5508-48ea-b9f0-afa3381d047d None admin] PATH_INFO: `/v3/roles/fa269b0e4dc54a24bb729398ce38231c` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:27.426447 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47a2c02-5508-48ea-b9f0-afa3381d047d None admin] RBAC: Authorizing `identity:get_role(role_id=fa269b0e4dc54a24bb729398ce38231c)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:27.427807 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47a2c02-5508-48ea-b9f0-afa3381d047d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:27.429116 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 416/846] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Oct 27 09:25:27 2022] GET /identity/v3/roles/fa269b0e4dc54a24bb729398ce38231c => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:27.434747 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec6957-abad-4177-81e3-89f3356ad6b6 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:27.484499 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:27.528874 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec6957-abad-4177-81e3-89f3356ad6b6 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:27.561276 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd1ecc-7c63-472f-9dad-28355f5d4958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:27.561825 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd1ecc-7c63-472f-9dad-28355f5d4958 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:27.562097 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd1ecc-7c63-472f-9dad-28355f5d4958 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:27.562360 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd1ecc-7c63-472f-9dad-28355f5d4958 None admin] PATH_INFO: `/v3/users/e5112784e7c2472ca80eb5eec2d05ad6` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:27.574690 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffd1ecc-7c63-472f-9dad-28355f5d4958 None admin] RBAC: Authorizing `identity:get_user(user_id=e5112784e7c2472ca80eb5eec2d05ad6)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:27.576086 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffd1ecc-7c63-472f-9dad-28355f5d4958 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:27.577650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 431/847] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Oct 27 09:25:27 2022] GET /identity/v3/users/e5112784e7c2472ca80eb5eec2d05ad6 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:27.583241 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47a2c02-5508-48ea-b9f0-afa3381d047d None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:27.591890 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:27.681233 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47a2c02-5508-48ea-b9f0-afa3381d047d None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:27.713188 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddfa9f0-3b0d-48fd-80fe-d5bf0c698de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:27.713791 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddfa9f0-3b0d-48fd-80fe-d5bf0c698de6 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:27.714068 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddfa9f0-3b0d-48fd-80fe-d5bf0c698de6 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:27.714337 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddfa9f0-3b0d-48fd-80fe-d5bf0c698de6 None admin] PATH_INFO: `/v3/projects/357196595fb544bdbe85994308761315` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:27.715437 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddfa9f0-3b0d-48fd-80fe-d5bf0c698de6 None admin] RBAC: Authorizing `identity:get_project(project_id=357196595fb544bdbe85994308761315)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:27.716875 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddfa9f0-3b0d-48fd-80fe-d5bf0c698de6 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:27.718725 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 417/848] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Oct 27 09:25:27 2022] GET /identity/v3/projects/357196595fb544bdbe85994308761315 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:27.725095 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd1ecc-7c63-472f-9dad-28355f5d4958 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:27.818483 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd1ecc-7c63-472f-9dad-28355f5d4958 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:27.829237 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:27.852883 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0dde7-d35c-446e-962b-6e9f7b8b912c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:27.853522 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0dde7-d35c-446e-962b-6e9f7b8b912c None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:27.853796 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0dde7-d35c-446e-962b-6e9f7b8b912c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:27.854109 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0dde7-d35c-446e-962b-6e9f7b8b912c None admin] PATH_INFO: `/v3/projects/357196595fb544bdbe85994308761315/users/e5112784e7c2472ca80eb5eec2d05ad6/roles/fa269b0e4dc54a24bb729398ce38231c` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:27.856187 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb0dde7-d35c-446e-962b-6e9f7b8b912c None admin] RBAC: Authorizing `identity:create_grant(project_id=357196595fb544bdbe85994308761315, user_id=e5112784e7c2472ca80eb5eec2d05ad6, role_id=fa269b0e4dc54a24bb729398ce38231c)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:27.857629 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb0dde7-d35c-446e-962b-6e9f7b8b912c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:27.869876 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 432/849] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Oct 27 09:25:27 2022] PUT /identity/v3/projects/357196595fb544bdbe85994308761315/users/e5112784e7c2472ca80eb5eec2d05ad6/roles/fa269b0e4dc54a24bb729398ce38231c => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:25:29.973057 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36bbff-09a4-4048-ba93-ae484141f0db None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:29.973583 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36bbff-09a4-4048-ba93-ae484141f0db None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:29.973583 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36bbff-09a4-4048-ba93-ae484141f0db None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:29.975445 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 418/850] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:25:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:25:29.980628 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d77df-74d9-48c5-8839-7c820cf36848 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:29.981003 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d77df-74d9-48c5-8839-7c820cf36848 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:29.981172 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d77df-74d9-48c5-8839-7c820cf36848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:29.999318 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f57d77df-74d9-48c5-8839-7c820cf36848 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:25:30.000495 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f57d77df-74d9-48c5-8839-7c820cf36848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:25:30.064061 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 433/851] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:25:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 09:25:30.335965 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db36bbff-09a4-4048-ba93-ae484141f0db None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:30.409964 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:30.435634 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db36bbff-09a4-4048-ba93-ae484141f0db None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:30.469136 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e1170-95d3-4ebb-acdb-40a0877c3b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:30.469748 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e1170-95d3-4ebb-acdb-40a0877c3b44 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:30.470026 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e1170-95d3-4ebb-acdb-40a0877c3b44 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:30.470293 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e1170-95d3-4ebb-acdb-40a0877c3b44 None admin] PATH_INFO: `/v3/roles/fa269b0e4dc54a24bb729398ce38231c` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:30.471659 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447e1170-95d3-4ebb-acdb-40a0877c3b44 None admin] RBAC: Authorizing `identity:get_role(role_id=fa269b0e4dc54a24bb729398ce38231c)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:30.473051 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447e1170-95d3-4ebb-acdb-40a0877c3b44 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:30.474372 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 419/852] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Oct 27 09:25:30 2022] GET /identity/v3/roles/fa269b0e4dc54a24bb729398ce38231c => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:30.480074 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57d77df-74d9-48c5-8839-7c820cf36848 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:30.571863 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:30.574759 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57d77df-74d9-48c5-8839-7c820cf36848 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:30.608845 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c670a478-5026-4d66-a957-a40f29de293b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:30.609456 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c670a478-5026-4d66-a957-a40f29de293b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:30.609731 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c670a478-5026-4d66-a957-a40f29de293b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:30.609998 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c670a478-5026-4d66-a957-a40f29de293b None admin] PATH_INFO: `/v3/users/e5112784e7c2472ca80eb5eec2d05ad6` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:30.611084 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c670a478-5026-4d66-a957-a40f29de293b None admin] RBAC: Authorizing `identity:get_user(user_id=e5112784e7c2472ca80eb5eec2d05ad6)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:30.612491 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c670a478-5026-4d66-a957-a40f29de293b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:30.614053 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 434/853] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Oct 27 09:25:30 2022] GET /identity/v3/users/e5112784e7c2472ca80eb5eec2d05ad6 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:30.619613 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e1170-95d3-4ebb-acdb-40a0877c3b44 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:30.716567 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e1170-95d3-4ebb-acdb-40a0877c3b44 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:30.748754 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a8f6c-bca8-4b11-b311-571d16f381a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:30.749341 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a8f6c-bca8-4b11-b311-571d16f381a4 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:30.749616 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a8f6c-bca8-4b11-b311-571d16f381a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:30.749881 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a8f6c-bca8-4b11-b311-571d16f381a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:30.761559 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:30.815697 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65a8f6c-bca8-4b11-b311-571d16f381a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:30.817154 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65a8f6c-bca8-4b11-b311-571d16f381a4 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:30.911450 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 420/854] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Oct 27 09:25:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 27 09:25:30.917234 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c670a478-5026-4d66-a957-a40f29de293b None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:31.011196 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c670a478-5026-4d66-a957-a40f29de293b None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:31.042565 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908fbe95-687c-45db-8b3a-fae4476ba50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:31.043169 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908fbe95-687c-45db-8b3a-fae4476ba50b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:31.043439 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908fbe95-687c-45db-8b3a-fae4476ba50b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:31.043700 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908fbe95-687c-45db-8b3a-fae4476ba50b None admin] PATH_INFO: `/v3/projects/357196595fb544bdbe85994308761315` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:31.044793 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908fbe95-687c-45db-8b3a-fae4476ba50b None admin] RBAC: Authorizing `identity:get_project(project_id=357196595fb544bdbe85994308761315)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:31.046181 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908fbe95-687c-45db-8b3a-fae4476ba50b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:31.047749 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 435/855] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Oct 27 09:25:30 2022] GET /identity/v3/projects/357196595fb544bdbe85994308761315 => generated 311 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:31.054535 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a8f6c-bca8-4b11-b311-571d16f381a4 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:31.096177 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:31.151607 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a8f6c-bca8-4b11-b311-571d16f381a4 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:31.184998 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7e78b6-0482-4c72-a2fb-2a2693a22e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:31.185532 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e78b6-0482-4c72-a2fb-2a2693a22e1c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:31.185806 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e78b6-0482-4c72-a2fb-2a2693a22e1c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:31.186071 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e78b6-0482-4c72-a2fb-2a2693a22e1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:31.186778 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7e78b6-0482-4c72-a2fb-2a2693a22e1c None admin] RBAC: Adding query filter params (role.id=fa269b0e4dc54a24bb729398ce38231c, scope.project.id=357196595fb544bdbe85994308761315, user.id=e5112784e7c2472ca80eb5eec2d05ad6) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:25:31.187085 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7e78b6-0482-4c72-a2fb-2a2693a22e1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:31.188502 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7e78b6-0482-4c72-a2fb-2a2693a22e1c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:31.194149 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 421/856] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Oct 27 09:25:31 2022] GET /identity/v3/role_assignments?user.id=e5112784e7c2472ca80eb5eec2d05ad6&scope.project.id=357196595fb544bdbe85994308761315&role.id=fa269b0e4dc54a24bb729398ce38231c => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:33.310135 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4242b60-fd4d-4f7b-8e03-bc91656c9a42 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:33.310559 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4242b60-fd4d-4f7b-8e03-bc91656c9a42 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:33.310661 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4242b60-fd4d-4f7b-8e03-bc91656c9a42 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:33.311994 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 436/857] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:25:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:25:33.317318 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d578dc-dd69-4a58-8bf9-8cc627370ed7 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:33.318280 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d578dc-dd69-4a58-8bf9-8cc627370ed7 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:33.318491 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d578dc-dd69-4a58-8bf9-8cc627370ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:33.337605 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-29d578dc-dd69-4a58-8bf9-8cc627370ed7 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:25:33.338818 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-29d578dc-dd69-4a58-8bf9-8cc627370ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:25:33.404193 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 422/858] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:25:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 09:25:33.669665 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4242b60-fd4d-4f7b-8e03-bc91656c9a42 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:33.704688 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:33.767238 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4242b60-fd4d-4f7b-8e03-bc91656c9a42 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:33.799187 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea221cc-bcf3-4219-b77d-b9a1d9390003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:33.799793 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea221cc-bcf3-4219-b77d-b9a1d9390003 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:33.800068 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea221cc-bcf3-4219-b77d-b9a1d9390003 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:33.800335 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea221cc-bcf3-4219-b77d-b9a1d9390003 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:33.801439 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea221cc-bcf3-4219-b77d-b9a1d9390003 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:33.803433 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea221cc-bcf3-4219-b77d-b9a1d9390003 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:33.804977 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 437/859] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:25:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:33.810914 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d578dc-dd69-4a58-8bf9-8cc627370ed7 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:33.844118 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:33.906010 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d578dc-dd69-4a58-8bf9-8cc627370ed7 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:33.938775 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd657b4b-afa0-4669-80d2-55105805669f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:33.939307 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd657b4b-afa0-4669-80d2-55105805669f None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:33.939578 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd657b4b-afa0-4669-80d2-55105805669f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:33.939841 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd657b4b-afa0-4669-80d2-55105805669f None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:33.940484 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd657b4b-afa0-4669-80d2-55105805669f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:33.941926 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd657b4b-afa0-4669-80d2-55105805669f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:33.968210 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 423/860] 10.222.0.70 () {60 vars in 1141 bytes} [Thu Oct 27 09:25:33 2022] POST /identity/v3/projects => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:25:36.132751 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ca896-b07b-4664-b2ad-f93780dac15c None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:36.133318 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ca896-b07b-4664-b2ad-f93780dac15c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:36.133318 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ca896-b07b-4664-b2ad-f93780dac15c None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:36.134612 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 438/861] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:25:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:25:36.140007 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2fb320-0514-4282-92e6-6c0e53f52963 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:36.140435 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2fb320-0514-4282-92e6-6c0e53f52963 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:36.140566 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2fb320-0514-4282-92e6-6c0e53f52963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:36.158625 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-ae2fb320-0514-4282-92e6-6c0e53f52963 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:25:36.159784 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ae2fb320-0514-4282-92e6-6c0e53f52963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:25:36.223213 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 424/862] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:25:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 09:25:36.488529 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010ca896-b07b-4664-b2ad-f93780dac15c None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:36.585176 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010ca896-b07b-4664-b2ad-f93780dac15c None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:36.587380 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:36.617381 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ea28c-b784-4ce8-8a6c-a8e8d0ff294c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:36.617982 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ea28c-b784-4ce8-8a6c-a8e8d0ff294c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:36.618254 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ea28c-b784-4ce8-8a6c-a8e8d0ff294c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:36.618521 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ea28c-b784-4ce8-8a6c-a8e8d0ff294c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:36.619595 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519ea28c-b784-4ce8-8a6c-a8e8d0ff294c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:36.621058 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519ea28c-b784-4ce8-8a6c-a8e8d0ff294c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:36.622582 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 439/863] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:25:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:36.628191 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2fb320-0514-4282-92e6-6c0e53f52963 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:36.646705 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:36.725944 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2fb320-0514-4282-92e6-6c0e53f52963 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:36.758960 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abcd1e7-cc23-44a2-b8d5-a038b98f205b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:36.759490 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abcd1e7-cc23-44a2-b8d5-a038b98f205b None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:36.759764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abcd1e7-cc23-44a2-b8d5-a038b98f205b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:36.760029 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abcd1e7-cc23-44a2-b8d5-a038b98f205b None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:36.760698 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abcd1e7-cc23-44a2-b8d5-a038b98f205b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:36.762120 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abcd1e7-cc23-44a2-b8d5-a038b98f205b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:36.783025 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 425/864] 10.222.0.70 () {60 vars in 1132 bytes} [Thu Oct 27 09:25:36 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:25:38.937298 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed9cd1f-e8d9-42cd-af50-8ca6e17f2bc2 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:38.937820 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed9cd1f-e8d9-42cd-af50-8ca6e17f2bc2 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:38.937820 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed9cd1f-e8d9-42cd-af50-8ca6e17f2bc2 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:38.939161 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 440/865] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:25:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:25:38.945031 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f649ae-c740-4c68-8ad6-c8f6396b89ee None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:38.945462 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f649ae-c740-4c68-8ad6-c8f6396b89ee None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:38.945555 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f649ae-c740-4c68-8ad6-c8f6396b89ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:38.963770 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-d7f649ae-c740-4c68-8ad6-c8f6396b89ee None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:25:38.964994 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d7f649ae-c740-4c68-8ad6-c8f6396b89ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:25:39.029926 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 426/866] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:25:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 09:25:39.296862 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed9cd1f-e8d9-42cd-af50-8ca6e17f2bc2 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:39.393340 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed9cd1f-e8d9-42cd-af50-8ca6e17f2bc2 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:39.425728 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e08cd-db95-465d-83fc-1427f44d90f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:39.426324 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e08cd-db95-465d-83fc-1427f44d90f0 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:39.426596 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e08cd-db95-465d-83fc-1427f44d90f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:39.426862 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e08cd-db95-465d-83fc-1427f44d90f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:39.436889 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e08cd-db95-465d-83fc-1427f44d90f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:25:39.441519 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-dc6e08cd-db95-465d-83fc-1427f44d90f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 09:25:39.441519 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:39.441519 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:25:39.441519 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:25:39.441519 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:25:39.441519 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:25:39.441519 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:25:39.441519 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:25:39.441519 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:25:39.441519 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:25:39.441519 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:25:39.441519 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:25:39.441519 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:25:39.441519 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:25:39.441519 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:25:39.441519 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:25:39.442581 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:25:39.442581 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:25:39.442581 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:25:39.442581 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:25:39.442581 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:25:39.442581 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:25:39.442581 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:25:39.442581 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:25:39.442581 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:25:39.442581 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:25:39.442581 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:25:39.442581 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:25:39.442581 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:25:39.442581 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:25:39.442581 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:25:39.442581 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:25:39.443430 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:25:39.443430 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:25:39.443430 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:25:39.443430 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:25:39.443430 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:25:39.443430 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:25:39.443430 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:25:39.443430 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:25:39.443430 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:25:39.443430 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:25:39.443430 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 09:25:39.443430 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:25:39.443430 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 441/867] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Oct 27 09:25:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:25:39.448431 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f649ae-c740-4c68-8ad6-c8f6396b89ee None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:39.451555 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:39.546110 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f649ae-c740-4c68-8ad6-c8f6396b89ee None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:39.578491 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dbc05a-0571-459f-87e6-49ca5ae4da79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:39.579077 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dbc05a-0571-459f-87e6-49ca5ae4da79 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:39.579353 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dbc05a-0571-459f-87e6-49ca5ae4da79 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:39.579619 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dbc05a-0571-459f-87e6-49ca5ae4da79 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:39.580259 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dbc05a-0571-459f-87e6-49ca5ae4da79 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:25:39.580584 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dbc05a-0571-459f-87e6-49ca5ae4da79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:39.581939 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dbc05a-0571-459f-87e6-49ca5ae4da79 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:39.589669 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 427/868] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Oct 27 09:25:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:39.595602 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e08cd-db95-465d-83fc-1427f44d90f0 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:39.626899 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:39.693060 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e08cd-db95-465d-83fc-1427f44d90f0 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:39.725932 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad9a50f-ddb1-4831-8433-110e706767f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:39.726543 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad9a50f-ddb1-4831-8433-110e706767f9 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:39.726820 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad9a50f-ddb1-4831-8433-110e706767f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:39.727090 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad9a50f-ddb1-4831-8433-110e706767f9 None admin] PATH_INFO: `/v3/users/d1309a8da4644d94a851da696e526f92` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:39.740127 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad9a50f-ddb1-4831-8433-110e706767f9 None admin] RBAC: Authorizing `identity:get_user(user_id=d1309a8da4644d94a851da696e526f92)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:39.741573 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad9a50f-ddb1-4831-8433-110e706767f9 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:39.743127 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 442/869] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Oct 27 09:25:39 2022] GET /identity/v3/users/d1309a8da4644d94a851da696e526f92 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:39.748753 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dbc05a-0571-459f-87e6-49ca5ae4da79 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:39.811896 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:39.847428 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dbc05a-0571-459f-87e6-49ca5ae4da79 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:39.879971 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f073b2-bbd1-452b-aebd-e7be4a584e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:39.880629 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f073b2-bbd1-452b-aebd-e7be4a584e38 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:39.880912 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f073b2-bbd1-452b-aebd-e7be4a584e38 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:39.881182 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f073b2-bbd1-452b-aebd-e7be4a584e38 None admin] PATH_INFO: `/v3/projects/9abf29f96a6d44eab77eac6e7468f950` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:39.882283 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f073b2-bbd1-452b-aebd-e7be4a584e38 None admin] RBAC: Authorizing `identity:get_project(project_id=9abf29f96a6d44eab77eac6e7468f950)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:39.883703 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f073b2-bbd1-452b-aebd-e7be4a584e38 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:39.885321 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 428/870] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Oct 27 09:25:39 2022] GET /identity/v3/projects/9abf29f96a6d44eab77eac6e7468f950 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:39.891153 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad9a50f-ddb1-4831-8433-110e706767f9 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:39.978122 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:39.987601 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad9a50f-ddb1-4831-8433-110e706767f9 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:40.020462 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde6aec2-d359-4899-a781-17f4ab77bd4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:40.021157 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde6aec2-d359-4899-a781-17f4ab77bd4b None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:40.021436 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde6aec2-d359-4899-a781-17f4ab77bd4b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:40.021707 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde6aec2-d359-4899-a781-17f4ab77bd4b None admin] PATH_INFO: `/v3/projects/9abf29f96a6d44eab77eac6e7468f950/users/d1309a8da4644d94a851da696e526f92/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:40.023558 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde6aec2-d359-4899-a781-17f4ab77bd4b None admin] RBAC: Authorizing `identity:create_grant(project_id=9abf29f96a6d44eab77eac6e7468f950, user_id=d1309a8da4644d94a851da696e526f92, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:40.025028 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde6aec2-d359-4899-a781-17f4ab77bd4b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:40.036755 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 443/871] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Oct 27 09:25:39 2022] PUT /identity/v3/projects/9abf29f96a6d44eab77eac6e7468f950/users/d1309a8da4644d94a851da696e526f92/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:25:42.123459 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3a790-3c59-4846-a568-da2e9a930ec3 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:42.123990 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3a790-3c59-4846-a568-da2e9a930ec3 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:42.123990 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3a790-3c59-4846-a568-da2e9a930ec3 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:42.125333 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 429/872] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:25:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:25:42.131077 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c7338-271c-4dcd-bca8-70e9465a4808 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:42.131493 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c7338-271c-4dcd-bca8-70e9465a4808 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:42.131625 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c7338-271c-4dcd-bca8-70e9465a4808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:42.149294 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-899c7338-271c-4dcd-bca8-70e9465a4808 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:25:42.150471 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-899c7338-271c-4dcd-bca8-70e9465a4808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:25:42.214172 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 444/873] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:25:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 27 09:25:42.480360 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf3a790-3c59-4846-a568-da2e9a930ec3 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:42.576865 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf3a790-3c59-4846-a568-da2e9a930ec3 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:42.609048 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ed0b0-c371-4bfd-a1cb-c87f13f4daf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:42.609651 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ed0b0-c371-4bfd-a1cb-c87f13f4daf2 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:42.609924 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ed0b0-c371-4bfd-a1cb-c87f13f4daf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:42.610190 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ed0b0-c371-4bfd-a1cb-c87f13f4daf2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:42.612005 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:42.621263 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ed0b0-c371-4bfd-a1cb-c87f13f4daf2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:25:42.625917 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-306ed0b0-c371-4bfd-a1cb-c87f13f4daf2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 09:25:42.625917 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:42.625917 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:25:42.625917 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:25:42.625917 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:25:42.625917 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:25:42.625917 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:25:42.625917 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:25:42.625917 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:25:42.625917 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:25:42.625917 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:25:42.625917 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:25:42.625917 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:25:42.625917 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:25:42.625917 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:25:42.625917 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:25:42.626989 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:25:42.626989 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:25:42.626989 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:25:42.626989 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:25:42.626989 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:25:42.626989 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:25:42.626989 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:25:42.626989 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:25:42.626989 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:25:42.626989 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:25:42.626989 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:25:42.626989 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:25:42.626989 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:25:42.626989 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:25:42.626989 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:25:42.626989 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:25:42.628156 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:25:42.628156 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:25:42.628156 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:25:42.628156 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:25:42.628156 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:25:42.628156 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:25:42.628156 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:25:42.628156 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:25:42.628156 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:25:42.628156 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:25:42.628156 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 09:25:42.628156 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:25:42.628156 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 430/874] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Oct 27 09:25:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:25:42.632999 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c7338-271c-4dcd-bca8-70e9465a4808 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:42.708502 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:42.728242 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c7338-271c-4dcd-bca8-70e9465a4808 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:42.760400 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5103861-91d8-41ca-870f-e4d440603b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:42.760965 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5103861-91d8-41ca-870f-e4d440603b04 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:42.761236 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5103861-91d8-41ca-870f-e4d440603b04 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:42.761501 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5103861-91d8-41ca-870f-e4d440603b04 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:42.762136 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5103861-91d8-41ca-870f-e4d440603b04 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:25:42.762430 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5103861-91d8-41ca-870f-e4d440603b04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:42.763762 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5103861-91d8-41ca-870f-e4d440603b04 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:42.771869 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 445/875] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Oct 27 09:25:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:25:42.777703 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ed0b0-c371-4bfd-a1cb-c87f13f4daf2 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:42.874314 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ed0b0-c371-4bfd-a1cb-c87f13f4daf2 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:42.906861 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a158f693-3b60-4757-b3bc-bf537b97583d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:42.907464 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158f693-3b60-4757-b3bc-bf537b97583d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:42.907740 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158f693-3b60-4757-b3bc-bf537b97583d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:42.908012 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158f693-3b60-4757-b3bc-bf537b97583d None admin] PATH_INFO: `/v3/users/d1309a8da4644d94a851da696e526f92` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:42.909405 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a158f693-3b60-4757-b3bc-bf537b97583d None admin] RBAC: Authorizing `identity:get_user(user_id=d1309a8da4644d94a851da696e526f92)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:42.911783 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a158f693-3b60-4757-b3bc-bf537b97583d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:42.913376 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 431/876] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Oct 27 09:25:42 2022] GET /identity/v3/users/d1309a8da4644d94a851da696e526f92 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:42.919037 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5103861-91d8-41ca-870f-e4d440603b04 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:43.015715 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5103861-91d8-41ca-870f-e4d440603b04 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:43.048211 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a88e07c-8b4c-4e2f-87e5-da8212c4c5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:43.048827 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88e07c-8b4c-4e2f-87e5-da8212c4c5bd None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:43.049105 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88e07c-8b4c-4e2f-87e5-da8212c4c5bd None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:43.049371 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88e07c-8b4c-4e2f-87e5-da8212c4c5bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:43.060594 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:43.113911 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a88e07c-8b4c-4e2f-87e5-da8212c4c5bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:43.115418 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a88e07c-8b4c-4e2f-87e5-da8212c4c5bd None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:43.212063 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 446/877] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Oct 27 09:25:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 27 09:25:43.218369 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a158f693-3b60-4757-b3bc-bf537b97583d None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:43.278180 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:43.317006 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a158f693-3b60-4757-b3bc-bf537b97583d None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:43.350126 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07edea86-43bd-4009-81ed-ba6e434fb2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:43.350738 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07edea86-43bd-4009-81ed-ba6e434fb2ed None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:43.351011 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07edea86-43bd-4009-81ed-ba6e434fb2ed None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:43.351278 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07edea86-43bd-4009-81ed-ba6e434fb2ed None admin] PATH_INFO: `/v3/projects/9abf29f96a6d44eab77eac6e7468f950` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:43.352367 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07edea86-43bd-4009-81ed-ba6e434fb2ed None admin] RBAC: Authorizing `identity:get_project(project_id=9abf29f96a6d44eab77eac6e7468f950)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:43.353819 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07edea86-43bd-4009-81ed-ba6e434fb2ed None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:43.355413 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 432/878] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Oct 27 09:25:43 2022] GET /identity/v3/projects/9abf29f96a6d44eab77eac6e7468f950 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:43.361550 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a88e07c-8b4c-4e2f-87e5-da8212c4c5bd None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:43.403152 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:43.456605 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a88e07c-8b4c-4e2f-87e5-da8212c4c5bd None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:43.489202 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c40649-7c94-445f-9632-3017aa3ea5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:43.489777 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c40649-7c94-445f-9632-3017aa3ea5fc None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:43.490055 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c40649-7c94-445f-9632-3017aa3ea5fc None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:43.490323 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c40649-7c94-445f-9632-3017aa3ea5fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:43.491029 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c40649-7c94-445f-9632-3017aa3ea5fc None admin] RBAC: Adding query filter params (role.id=0dfd839b7a194aee96f82f4ebc1620a7, scope.project.id=9abf29f96a6d44eab77eac6e7468f950, user.id=d1309a8da4644d94a851da696e526f92) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:25:43.491333 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c40649-7c94-445f-9632-3017aa3ea5fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:43.492758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c40649-7c94-445f-9632-3017aa3ea5fc None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:43.498275 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 447/879] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Oct 27 09:25:43 2022] GET /identity/v3/role_assignments?user.id=d1309a8da4644d94a851da696e526f92&scope.project.id=9abf29f96a6d44eab77eac6e7468f950&role.id=0dfd839b7a194aee96f82f4ebc1620a7 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:25:45.635954 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddaf03d-11bd-4657-a25a-af88a20a177b None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:45.636467 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddaf03d-11bd-4657-a25a-af88a20a177b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:45.636467 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddaf03d-11bd-4657-a25a-af88a20a177b None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:45.637846 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 433/880] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:25:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:25:45.643109 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620e04ec-3987-4f5a-9d71-1653d7c7f0dc None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:45.643531 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620e04ec-3987-4f5a-9d71-1653d7c7f0dc None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:45.643650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620e04ec-3987-4f5a-9d71-1653d7c7f0dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:45.661739 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-620e04ec-3987-4f5a-9d71-1653d7c7f0dc None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:25:45.662900 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-620e04ec-3987-4f5a-9d71-1653d7c7f0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:25:45.725693 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 448/881] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:25:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 09:25:45.990714 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddaf03d-11bd-4657-a25a-af88a20a177b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:46.088352 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddaf03d-11bd-4657-a25a-af88a20a177b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:46.116471 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:46.122583 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b4f138-367c-4f68-8e3f-cdd6fce85301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:46.123169 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4f138-367c-4f68-8e3f-cdd6fce85301 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:46.123444 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4f138-367c-4f68-8e3f-cdd6fce85301 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:46.123712 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4f138-367c-4f68-8e3f-cdd6fce85301 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:46.124368 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b4f138-367c-4f68-8e3f-cdd6fce85301 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:46.125750 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b4f138-367c-4f68-8e3f-cdd6fce85301 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:46.140235 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 434/882] 10.222.0.70 () {60 vars in 1138 bytes} [Thu Oct 27 09:25:45 2022] POST /identity/v3/domains => generated 267 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:25:48.295838 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f1700e-740e-4e01-855d-ef2ce58cd242 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:48.296313 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f1700e-740e-4e01-855d-ef2ce58cd242 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:48.296410 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f1700e-740e-4e01-855d-ef2ce58cd242 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:48.297736 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 449/883] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:25:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:25:48.303381 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0e481-7539-43f6-8d97-b23d3abc59d8 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:48.303777 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0e481-7539-43f6-8d97-b23d3abc59d8 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:48.303922 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0e481-7539-43f6-8d97-b23d3abc59d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:48.322120 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-5bf0e481-7539-43f6-8d97-b23d3abc59d8 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:25:48.323290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5bf0e481-7539-43f6-8d97-b23d3abc59d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:25:48.388236 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 435/884] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:25:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 09:25:48.658761 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f1700e-740e-4e01-855d-ef2ce58cd242 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:48.694678 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:48.757793 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f1700e-740e-4e01-855d-ef2ce58cd242 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:48.790106 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7d74e7-6f44-4827-976a-155fe7931a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:48.790706 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d74e7-6f44-4827-976a-155fe7931a03 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:48.790976 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d74e7-6f44-4827-976a-155fe7931a03 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:48.791240 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d74e7-6f44-4827-976a-155fe7931a03 None admin] PATH_INFO: `/v3/domains/5975686c79ed4769987db1e099c1940e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:48.801166 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7d74e7-6f44-4827-976a-155fe7931a03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=5975686c79ed4769987db1e099c1940e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:48.802580 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7d74e7-6f44-4827-976a-155fe7931a03 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:48.804115 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 450/885] 10.222.0.70 () {56 vars in 1183 bytes} [Thu Oct 27 09:25:48 2022] GET /identity/v3/domains/5975686c79ed4769987db1e099c1940e => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:48.809845 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf0e481-7539-43f6-8d97-b23d3abc59d8 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:48.904026 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf0e481-7539-43f6-8d97-b23d3abc59d8 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:48.924735 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:48.938961 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0620cf-4ba7-46da-ba01-063e523703d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:48.939561 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0620cf-4ba7-46da-ba01-063e523703d5 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:48.939839 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0620cf-4ba7-46da-ba01-063e523703d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:48.940109 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0620cf-4ba7-46da-ba01-063e523703d5 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:48.940792 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0620cf-4ba7-46da-ba01-063e523703d5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:48.942218 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0620cf-4ba7-46da-ba01-063e523703d5 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:48.967693 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 436/886] 10.222.0.70 () {60 vars in 1141 bytes} [Thu Oct 27 09:25:48 2022] POST /identity/v3/projects => generated 361 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:25:51.122591 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b491c7b8-66bc-4240-9b66-2c38be455688 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:51.123128 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b491c7b8-66bc-4240-9b66-2c38be455688 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:51.123128 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b491c7b8-66bc-4240-9b66-2c38be455688 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:51.124600 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 451/887] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:25:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:25:51.130462 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c403f-c223-4a4c-aaa8-c7080d1f64c5 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:51.130893 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c403f-c223-4a4c-aaa8-c7080d1f64c5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:51.131009 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c403f-c223-4a4c-aaa8-c7080d1f64c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:51.149322 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-350c403f-c223-4a4c-aaa8-c7080d1f64c5 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:25:51.150659 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-350c403f-c223-4a4c-aaa8-c7080d1f64c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:25:51.216183 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 437/888] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:25:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 09:25:51.485468 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b491c7b8-66bc-4240-9b66-2c38be455688 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:51.576337 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:51.583649 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b491c7b8-66bc-4240-9b66-2c38be455688 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:51.615480 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488aae9b-0347-4c59-8328-37efaac9c055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:51.616093 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488aae9b-0347-4c59-8328-37efaac9c055 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:51.616372 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488aae9b-0347-4c59-8328-37efaac9c055 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:51.616674 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488aae9b-0347-4c59-8328-37efaac9c055 None admin] PATH_INFO: `/v3/domains/5975686c79ed4769987db1e099c1940e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:51.617770 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488aae9b-0347-4c59-8328-37efaac9c055 None admin] RBAC: Authorizing `identity:get_domain(domain_id=5975686c79ed4769987db1e099c1940e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:51.619177 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488aae9b-0347-4c59-8328-37efaac9c055 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:51.620736 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 452/889] 10.222.0.70 () {56 vars in 1183 bytes} [Thu Oct 27 09:25:51 2022] GET /identity/v3/domains/5975686c79ed4769987db1e099c1940e => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:51.626312 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c403f-c223-4a4c-aaa8-c7080d1f64c5 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:51.722467 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c403f-c223-4a4c-aaa8-c7080d1f64c5 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:51.723966 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:51.757246 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023779d6-fc90-42a2-bf3d-ad69138ea1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:51.757774 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023779d6-fc90-42a2-bf3d-ad69138ea1be None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:51.758048 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023779d6-fc90-42a2-bf3d-ad69138ea1be None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:51.758312 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023779d6-fc90-42a2-bf3d-ad69138ea1be None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:51.758953 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023779d6-fc90-42a2-bf3d-ad69138ea1be None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:51.760367 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023779d6-fc90-42a2-bf3d-ad69138ea1be None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:51.780586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 438/890] 10.222.0.70 () {60 vars in 1132 bytes} [Thu Oct 27 09:25:51 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:25:53.940249 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e94b4f-4e93-47dc-861b-e9b0415e8deb None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:53.940836 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e94b4f-4e93-47dc-861b-e9b0415e8deb None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:53.940836 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e94b4f-4e93-47dc-861b-e9b0415e8deb None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:53.942144 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 453/891] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:25:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:25:53.947423 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c3de8-89f6-4af7-843a-d12a7ea6bce8 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:53.947837 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c3de8-89f6-4af7-843a-d12a7ea6bce8 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:53.947969 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c3de8-89f6-4af7-843a-d12a7ea6bce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:53.966587 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-483c3de8-89f6-4af7-843a-d12a7ea6bce8 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:25:53.967768 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-483c3de8-89f6-4af7-843a-d12a7ea6bce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:25:54.032921 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 439/892] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:25:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 09:25:54.296848 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e94b4f-4e93-47dc-861b-e9b0415e8deb None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:54.395752 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e94b4f-4e93-47dc-861b-e9b0415e8deb None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:54.429011 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a38cc5a-486b-497a-88f6-0fb1b34d8e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:54.429618 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a38cc5a-486b-497a-88f6-0fb1b34d8e93 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:54.429892 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a38cc5a-486b-497a-88f6-0fb1b34d8e93 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:54.430158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a38cc5a-486b-497a-88f6-0fb1b34d8e93 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:54.440123 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a38cc5a-486b-497a-88f6-0fb1b34d8e93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:25:54.444882 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-4a38cc5a-486b-497a-88f6-0fb1b34d8e93 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 09:25:54.444882 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:54.444882 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:25:54.444882 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:25:54.444882 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:25:54.444882 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:25:54.444882 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:25:54.444882 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:25:54.444882 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:25:54.444882 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:25:54.444882 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:25:54.444882 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:25:54.444882 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:25:54.444882 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:25:54.444882 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:25:54.444882 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:25:54.446025 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:25:54.446025 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:25:54.446025 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:25:54.446025 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:25:54.446025 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:25:54.446025 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:25:54.446025 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:25:54.446025 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:25:54.446025 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:25:54.446025 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:25:54.446025 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:25:54.446025 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:25:54.446025 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:25:54.446025 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:25:54.446025 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:25:54.446025 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:25:54.446935 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:25:54.446935 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:25:54.446935 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:25:54.446935 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:25:54.446935 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:25:54.446935 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:25:54.446935 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:25:54.446935 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:25:54.446935 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:25:54.446935 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:25:54.446935 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 09:25:54.446935 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:25:54.446935 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 454/893] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Oct 27 09:25:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:25:54.451774 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483c3de8-89f6-4af7-843a-d12a7ea6bce8 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:54.536468 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:54.548616 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483c3de8-89f6-4af7-843a-d12a7ea6bce8 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:54.581683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db4d8a9-ecbf-4bb5-8fe4-4785b26c18e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:54.582267 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4d8a9-ecbf-4bb5-8fe4-4785b26c18e8 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:54.582537 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4d8a9-ecbf-4bb5-8fe4-4785b26c18e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:54.582797 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4d8a9-ecbf-4bb5-8fe4-4785b26c18e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:54.583438 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db4d8a9-ecbf-4bb5-8fe4-4785b26c18e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:25:54.583733 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db4d8a9-ecbf-4bb5-8fe4-4785b26c18e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:54.585140 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db4d8a9-ecbf-4bb5-8fe4-4785b26c18e8 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:54.593223 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 440/894] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Oct 27 09:25:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:54.599416 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a38cc5a-486b-497a-88f6-0fb1b34d8e93 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:54.624294 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:54.696044 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a38cc5a-486b-497a-88f6-0fb1b34d8e93 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:54.728341 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63486f9c-14b3-4079-8082-ab7e10e874a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:54.728973 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63486f9c-14b3-4079-8082-ab7e10e874a2 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:54.729249 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63486f9c-14b3-4079-8082-ab7e10e874a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:54.729516 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63486f9c-14b3-4079-8082-ab7e10e874a2 None admin] PATH_INFO: `/v3/users/3c00a7b501ef416ba55e0af63fcac6d0` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:54.741920 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63486f9c-14b3-4079-8082-ab7e10e874a2 None admin] RBAC: Authorizing `identity:get_user(user_id=3c00a7b501ef416ba55e0af63fcac6d0)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:54.743327 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63486f9c-14b3-4079-8082-ab7e10e874a2 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:54.744916 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 455/895] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Oct 27 09:25:54 2022] GET /identity/v3/users/3c00a7b501ef416ba55e0af63fcac6d0 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:54.750499 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db4d8a9-ecbf-4bb5-8fe4-4785b26c18e8 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:54.845393 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db4d8a9-ecbf-4bb5-8fe4-4785b26c18e8 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:54.877060 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6858d582-ca95-44f8-91ec-685a3ffc8f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:54.877666 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858d582-ca95-44f8-91ec-685a3ffc8f19 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:54.877937 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858d582-ca95-44f8-91ec-685a3ffc8f19 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:54.878202 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858d582-ca95-44f8-91ec-685a3ffc8f19 None admin] PATH_INFO: `/v3/projects/7456d8544f96482b8adf7284e842b3cf` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:54.879282 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6858d582-ca95-44f8-91ec-685a3ffc8f19 None admin] RBAC: Authorizing `identity:get_project(project_id=7456d8544f96482b8adf7284e842b3cf)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:54.881249 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6858d582-ca95-44f8-91ec-685a3ffc8f19 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:54.882816 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 441/896] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Oct 27 09:25:54 2022] GET /identity/v3/projects/7456d8544f96482b8adf7284e842b3cf => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:54.888641 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63486f9c-14b3-4079-8082-ab7e10e874a2 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:54.966727 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:54.984597 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63486f9c-14b3-4079-8082-ab7e10e874a2 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:55.017646 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0a44d8-620e-4fb8-96cb-c6a0ac06d096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:55.018282 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a44d8-620e-4fb8-96cb-c6a0ac06d096 None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:55.018560 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a44d8-620e-4fb8-96cb-c6a0ac06d096 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:55.018831 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a44d8-620e-4fb8-96cb-c6a0ac06d096 None admin] PATH_INFO: `/v3/projects/7456d8544f96482b8adf7284e842b3cf/users/3c00a7b501ef416ba55e0af63fcac6d0/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:55.021225 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0a44d8-620e-4fb8-96cb-c6a0ac06d096 None admin] RBAC: Authorizing `identity:create_grant(project_id=7456d8544f96482b8adf7284e842b3cf, user_id=3c00a7b501ef416ba55e0af63fcac6d0, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:55.022642 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0a44d8-620e-4fb8-96cb-c6a0ac06d096 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:55.034430 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 456/897] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Oct 27 09:25:54 2022] PUT /identity/v3/projects/7456d8544f96482b8adf7284e842b3cf/users/3c00a7b501ef416ba55e0af63fcac6d0/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:25:57.162731 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9093620e-ec52-45d0-86e4-ea4f7692e551 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:57.163276 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9093620e-ec52-45d0-86e4-ea4f7692e551 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:57.163276 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9093620e-ec52-45d0-86e4-ea4f7692e551 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:57.164617 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 442/898] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:25:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:25:57.169879 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0298c1-1b66-4182-9589-200c4076951e None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:57.170332 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0298c1-1b66-4182-9589-200c4076951e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:57.170435 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0298c1-1b66-4182-9589-200c4076951e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:57.190468 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-4f0298c1-1b66-4182-9589-200c4076951e None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:25:57.191662 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4f0298c1-1b66-4182-9589-200c4076951e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:25:57.255539 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 457/899] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:25:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 09:25:57.523045 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9093620e-ec52-45d0-86e4-ea4f7692e551 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:57.541900 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:57.621030 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9093620e-ec52-45d0-86e4-ea4f7692e551 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:57.653912 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04964224-1f0f-441c-99a2-a6b909003817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:57.654518 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04964224-1f0f-441c-99a2-a6b909003817 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:57.654795 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04964224-1f0f-441c-99a2-a6b909003817 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:57.655063 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04964224-1f0f-441c-99a2-a6b909003817 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:57.664871 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04964224-1f0f-441c-99a2-a6b909003817 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:25:57.669577 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-04964224-1f0f-441c-99a2-a6b909003817 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 09:25:57.669577 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:25:57.669577 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:25:57.669577 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:25:57.669577 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:25:57.669577 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:25:57.669577 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:25:57.669577 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:25:57.669577 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:25:57.669577 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:25:57.669577 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:25:57.669577 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:25:57.669577 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:25:57.669577 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:25:57.669577 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:25:57.669577 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:25:57.670670 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:25:57.670670 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:25:57.670670 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:25:57.670670 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:25:57.670670 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:25:57.670670 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:25:57.670670 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:25:57.670670 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:25:57.670670 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:25:57.670670 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:25:57.670670 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:25:57.670670 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:25:57.670670 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:25:57.670670 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:25:57.670670 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:25:57.670670 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:25:57.671703 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:25:57.671703 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:25:57.671703 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:25:57.671703 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:25:57.671703 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:25:57.671703 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:25:57.671703 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:25:57.671703 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:25:57.671703 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:25:57.671703 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:25:57.671703 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 09:25:57.671703 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:25:57.671703 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 443/900] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Oct 27 09:25:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:25:57.676695 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0298c1-1b66-4182-9589-200c4076951e None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:57.741917 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:57.771411 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0298c1-1b66-4182-9589-200c4076951e None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:57.804135 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d42fe-641a-4732-b18e-a96bed6f9980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:57.804705 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d42fe-641a-4732-b18e-a96bed6f9980 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:57.804986 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d42fe-641a-4732-b18e-a96bed6f9980 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:57.805254 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d42fe-641a-4732-b18e-a96bed6f9980 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:57.805898 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48d42fe-641a-4732-b18e-a96bed6f9980 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:25:57.806200 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48d42fe-641a-4732-b18e-a96bed6f9980 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:57.807552 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48d42fe-641a-4732-b18e-a96bed6f9980 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:57.815255 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 458/901] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Oct 27 09:25:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:57.821688 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04964224-1f0f-441c-99a2-a6b909003817 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:57.899225 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:57.920860 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04964224-1f0f-441c-99a2-a6b909003817 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:57.954209 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50f7a67-6130-4268-b6cf-69e5358b55a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:57.954822 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f7a67-6130-4268-b6cf-69e5358b55a0 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:57.955104 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f7a67-6130-4268-b6cf-69e5358b55a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:57.955380 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f7a67-6130-4268-b6cf-69e5358b55a0 None admin] PATH_INFO: `/v3/users/3c00a7b501ef416ba55e0af63fcac6d0` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:57.956485 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50f7a67-6130-4268-b6cf-69e5358b55a0 None admin] RBAC: Authorizing `identity:get_user(user_id=3c00a7b501ef416ba55e0af63fcac6d0)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:57.957962 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50f7a67-6130-4268-b6cf-69e5358b55a0 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:57.959509 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 444/902] 10.222.0.70 () {56 vars in 1177 bytes} [Thu Oct 27 09:25:57 2022] GET /identity/v3/users/3c00a7b501ef416ba55e0af63fcac6d0 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:57.965229 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d42fe-641a-4732-b18e-a96bed6f9980 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:58.060975 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d42fe-641a-4732-b18e-a96bed6f9980 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:58.092641 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe215081-2d52-49a6-a88b-eba41789a93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:58.093471 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe215081-2d52-49a6-a88b-eba41789a93f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:58.093742 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe215081-2d52-49a6-a88b-eba41789a93f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:58.094005 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe215081-2d52-49a6-a88b-eba41789a93f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:58.096455 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:58.159348 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe215081-2d52-49a6-a88b-eba41789a93f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:58.161325 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe215081-2d52-49a6-a88b-eba41789a93f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:58.254150 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 459/903] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Oct 27 09:25:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 27 09:25:58.259947 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50f7a67-6130-4268-b6cf-69e5358b55a0 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:58.358450 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50f7a67-6130-4268-b6cf-69e5358b55a0 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:58.390933 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97629d0-7211-48ca-9eca-7aa21e4c1730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:58.391548 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97629d0-7211-48ca-9eca-7aa21e4c1730 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:58.391824 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97629d0-7211-48ca-9eca-7aa21e4c1730 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:58.392092 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97629d0-7211-48ca-9eca-7aa21e4c1730 None admin] PATH_INFO: `/v3/projects/7456d8544f96482b8adf7284e842b3cf` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:58.393201 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97629d0-7211-48ca-9eca-7aa21e4c1730 None admin] RBAC: Authorizing `identity:get_project(project_id=7456d8544f96482b8adf7284e842b3cf)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:58.394631 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97629d0-7211-48ca-9eca-7aa21e4c1730 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:58.396606 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 445/904] 10.222.0.70 () {56 vars in 1186 bytes} [Thu Oct 27 09:25:58 2022] GET /identity/v3/projects/7456d8544f96482b8adf7284e842b3cf => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:25:58.402729 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe215081-2d52-49a6-a88b-eba41789a93f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:25:58.435944 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:25:58.500224 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe215081-2d52-49a6-a88b-eba41789a93f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:25:58.533001 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c9ec19-bfcf-41eb-a6d3-02361d74c888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:25:58.533585 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c9ec19-bfcf-41eb-a6d3-02361d74c888 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:25:58.533861 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c9ec19-bfcf-41eb-a6d3-02361d74c888 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:25:58.534129 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c9ec19-bfcf-41eb-a6d3-02361d74c888 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:25:58.534829 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c9ec19-bfcf-41eb-a6d3-02361d74c888 None admin] RBAC: Adding query filter params (role.id=0dfd839b7a194aee96f82f4ebc1620a7, scope.project.id=7456d8544f96482b8adf7284e842b3cf, user.id=3c00a7b501ef416ba55e0af63fcac6d0) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:25:58.535135 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c9ec19-bfcf-41eb-a6d3-02361d74c888 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:25:58.537167 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c9ec19-bfcf-41eb-a6d3-02361d74c888 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:25:58.542715 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 460/905] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Oct 27 09:25:58 2022] GET /identity/v3/role_assignments?user.id=3c00a7b501ef416ba55e0af63fcac6d0&scope.project.id=7456d8544f96482b8adf7284e842b3cf&role.id=0dfd839b7a194aee96f82f4ebc1620a7 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:26:08.386897 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=93 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:26:08.540235 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=51 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:26:53.761427 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0a5a6-ece1-4a99-aa77-514508577099 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:26:53.762026 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0a5a6-ece1-4a99-aa77-514508577099 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:26:53.762026 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0a5a6-ece1-4a99-aa77-514508577099 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:26:53.763291 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 446/906] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:26:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:26:53.769212 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91576ce9-31bc-45bc-869c-17d06ea317fe None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:26:53.769667 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91576ce9-31bc-45bc-869c-17d06ea317fe None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:26:53.769755 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91576ce9-31bc-45bc-869c-17d06ea317fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:26:53.788786 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-91576ce9-31bc-45bc-869c-17d06ea317fe None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:26:53.790305 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-91576ce9-31bc-45bc-869c-17d06ea317fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:26:53.864672 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 461/907] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:26:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 09:26:54.148115 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e0a5a6-ece1-4a99-aa77-514508577099 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:26:54.268570 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e0a5a6-ece1-4a99-aa77-514508577099 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:26:54.310562 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6439704-9055-458c-8e97-df3be63eeccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:26:54.311589 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6439704-9055-458c-8e97-df3be63eeccf None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:26:54.312083 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6439704-9055-458c-8e97-df3be63eeccf None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:26:54.312631 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6439704-9055-458c-8e97-df3be63eeccf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:26:54.329564 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6439704-9055-458c-8e97-df3be63eeccf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:26:54.331426 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6439704-9055-458c-8e97-df3be63eeccf None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:26:54.360467 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-f6439704-9055-458c-8e97-df3be63eeccf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:26:54.360467 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:26:54.360467 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:26:54.360467 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:26:54.360467 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:26:54.360467 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:26:54.360467 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:26:54.360467 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:26:54.360467 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:26:54.360467 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:26:54.360467 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:26:54.360467 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:26:54.360467 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:26:54.360467 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:26:54.360467 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:26:54.360467 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:26:54.362673 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:26:54.362673 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:26:54.362673 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:26:54.362673 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:26:54.362673 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:26:54.362673 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:26:54.362673 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:26:54.362673 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:26:54.362673 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:26:54.362673 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:26:54.362673 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:26:54.362673 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:26:54.362673 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:26:54.362673 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:26:54.362673 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:26:54.362673 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:26:54.362673 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:26:54.363749 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:26:54.363749 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:26:54.363749 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:26:54.363749 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 447/908] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:26:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:26:54.368770 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91576ce9-31bc-45bc-869c-17d06ea317fe None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:26:54.494919 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91576ce9-31bc-45bc-869c-17d06ea317fe None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:26:54.531470 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:26:54.540252 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad439df-62a6-49a1-a95e-95e3f96a6304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:26:54.540893 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad439df-62a6-49a1-a95e-95e3f96a6304 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:26:54.541171 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad439df-62a6-49a1-a95e-95e3f96a6304 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:26:54.541443 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad439df-62a6-49a1-a95e-95e3f96a6304 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:26:54.542085 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad439df-62a6-49a1-a95e-95e3f96a6304 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:26:54.542389 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad439df-62a6-49a1-a95e-95e3f96a6304 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:26:54.544309 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad439df-62a6-49a1-a95e-95e3f96a6304 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:26:54.567502 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 462/909] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:26:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:26:54.568826 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6439704-9055-458c-8e97-df3be63eeccf None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:26:54.638001 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:26:54.709584 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6439704-9055-458c-8e97-df3be63eeccf None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:26:54.754566 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e5c42-5bf4-4b86-9e84-da8f56046901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:26:54.755429 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e5c42-5bf4-4b86-9e84-da8f56046901 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:26:54.755865 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e5c42-5bf4-4b86-9e84-da8f56046901 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:26:54.756288 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e5c42-5bf4-4b86-9e84-da8f56046901 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:26:54.757141 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5e5c42-5bf4-4b86-9e84-da8f56046901 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:26:54.759080 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5e5c42-5bf4-4b86-9e84-da8f56046901 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:26:54.792563 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 448/910] 10.222.0.70 () {60 vars in 1132 bytes} [Thu Oct 27 09:26:54 2022] POST /identity/v3/users => generated 251 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:26:57.175497 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ff0bd-0739-46b0-af8f-8b3c77f5063e None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:26:57.176052 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ff0bd-0739-46b0-af8f-8b3c77f5063e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:26:57.176052 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ff0bd-0739-46b0-af8f-8b3c77f5063e None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:26:57.177441 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 463/911] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:26:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:26:57.182712 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d006c9a-f571-40e0-bc92-d9be031e1db8 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:26:57.183219 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d006c9a-f571-40e0-bc92-d9be031e1db8 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:26:57.183306 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d006c9a-f571-40e0-bc92-d9be031e1db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:26:57.202714 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-7d006c9a-f571-40e0-bc92-d9be031e1db8 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:26:57.203911 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7d006c9a-f571-40e0-bc92-d9be031e1db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:26:57.287276 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 449/912] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:26:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 09:26:57.557344 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1ff0bd-0739-46b0-af8f-8b3c77f5063e None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:26:57.671721 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1ff0bd-0739-46b0-af8f-8b3c77f5063e None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:26:57.707978 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1750b786-7481-446c-a722-68da78eab88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:26:57.708639 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750b786-7481-446c-a722-68da78eab88a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:26:57.708956 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750b786-7481-446c-a722-68da78eab88a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:26:57.709296 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750b786-7481-446c-a722-68da78eab88a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:26:57.720850 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1750b786-7481-446c-a722-68da78eab88a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:26:57.726148 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-1750b786-7481-446c-a722-68da78eab88a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 27 09:26:57.726148 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:26:57.726148 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:26:57.726148 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:26:57.726148 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:26:57.726148 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:26:57.726148 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:26:57.726148 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:26:57.726148 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:26:57.726148 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:26:57.726148 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:26:57.726148 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:26:57.726148 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:26:57.726148 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:26:57.726148 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:26:57.726148 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:26:57.727163 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:26:57.727163 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:26:57.727163 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:26:57.727163 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:26:57.727163 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:26:57.727163 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:26:57.727163 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:26:57.727163 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:26:57.727163 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:26:57.727163 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:26:57.727163 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:26:57.727163 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:26:57.727163 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:26:57.727163 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:26:57.727163 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:26:57.727163 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:26:57.728018 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:26:57.728018 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:26:57.728018 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:26:57.728018 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:26:57.728018 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:26:57.728018 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:26:57.728018 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:26:57.728018 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:26:57.728018 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:26:57.728018 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:26:57.728018 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 27 09:26:57.728018 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:26:57.728018 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 464/913] 10.222.0.70 () {56 vars in 1102 bytes} [Thu Oct 27 09:26:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:26:57.733373 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d006c9a-f571-40e0-bc92-d9be031e1db8 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:26:57.775808 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:26:57.840631 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d006c9a-f571-40e0-bc92-d9be031e1db8 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:26:57.880004 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a803737-4200-448a-8c6f-9b42a25c290d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:26:57.880706 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a803737-4200-448a-8c6f-9b42a25c290d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:26:57.880990 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a803737-4200-448a-8c6f-9b42a25c290d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:26:57.881287 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a803737-4200-448a-8c6f-9b42a25c290d None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:26:57.882066 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a803737-4200-448a-8c6f-9b42a25c290d None admin] RBAC: Adding query filter params (name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:26:57.882322 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a803737-4200-448a-8c6f-9b42a25c290d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:26:57.883801 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a803737-4200-448a-8c6f-9b42a25c290d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:26:57.895999 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 450/914] 10.222.0.70 () {56 vars in 1103 bytes} [Thu Oct 27 09:26:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:26:57.901517 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1750b786-7481-446c-a722-68da78eab88a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:26:57.959591 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:26:58.021401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1750b786-7481-446c-a722-68da78eab88a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:26:58.060175 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be162e5-dd2c-4398-ae7e-11afd8ce9291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:26:58.061143 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be162e5-dd2c-4398-ae7e-11afd8ce9291 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:26:58.061422 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be162e5-dd2c-4398-ae7e-11afd8ce9291 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:26:58.061698 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be162e5-dd2c-4398-ae7e-11afd8ce9291 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:26:58.074437 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be162e5-dd2c-4398-ae7e-11afd8ce9291 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:26:58.075860 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be162e5-dd2c-4398-ae7e-11afd8ce9291 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:26:58.092439 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-1be162e5-dd2c-4398-ae7e-11afd8ce9291 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:26:58.092439 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:26:58.092439 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:26:58.092439 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:26:58.092439 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:26:58.092439 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:26:58.092439 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:26:58.092439 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:26:58.092439 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:26:58.092439 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:26:58.092439 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:26:58.092439 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:26:58.092439 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:26:58.092439 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:26:58.092439 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:26:58.092439 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:26:58.096728 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:26:58.096728 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:26:58.096728 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:26:58.096728 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:26:58.096728 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:26:58.096728 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:26:58.096728 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:26:58.096728 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:26:58.096728 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:26:58.096728 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:26:58.096728 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:26:58.096728 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:26:58.096728 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:26:58.096728 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:26:58.096728 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:26:58.096728 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:26:58.096728 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:26:58.097823 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:26:58.097823 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:26:58.097823 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:26:58.097823 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 465/915] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:26:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:26:58.099953 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a803737-4200-448a-8c6f-9b42a25c290d None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:26:58.208776 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:26:58.211906 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a803737-4200-448a-8c6f-9b42a25c290d None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:26:58.252350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397dfeb0-79ba-4eea-9254-3f504bb2c4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:26:58.254202 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397dfeb0-79ba-4eea-9254-3f504bb2c4b4 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:26:58.254545 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397dfeb0-79ba-4eea-9254-3f504bb2c4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:26:58.254852 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397dfeb0-79ba-4eea-9254-3f504bb2c4b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:26:58.255491 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397dfeb0-79ba-4eea-9254-3f504bb2c4b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:26:58.255794 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397dfeb0-79ba-4eea-9254-3f504bb2c4b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:26:58.257276 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397dfeb0-79ba-4eea-9254-3f504bb2c4b4 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:26:58.270590 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 451/916] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:26:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:26:58.275987 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be162e5-dd2c-4398-ae7e-11afd8ce9291 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:26:58.388091 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be162e5-dd2c-4398-ae7e-11afd8ce9291 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:26:58.389937 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:26:58.425722 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f02dd6-ffdd-4417-9788-5cc4e45ec138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:26:58.426558 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f02dd6-ffdd-4417-9788-5cc4e45ec138 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:26:58.426857 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f02dd6-ffdd-4417-9788-5cc4e45ec138 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:26:58.427126 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f02dd6-ffdd-4417-9788-5cc4e45ec138 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:26:58.435093 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f02dd6-ffdd-4417-9788-5cc4e45ec138 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:26:58.436510 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f02dd6-ffdd-4417-9788-5cc4e45ec138 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:26:58.448548 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-23f02dd6-ffdd-4417-9788-5cc4e45ec138 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 27 09:26:58.448548 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:26:58.448548 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:26:58.448548 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:26:58.448548 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:26:58.448548 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:26:58.448548 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:26:58.448548 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:26:58.448548 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:26:58.448548 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:26:58.448548 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:26:58.448548 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:26:58.448548 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 09:26:58.448548 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 09:26:58.448548 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 09:26:58.448548 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 09:26:58.449528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:26:58.449528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:26:58.449528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 09:26:58.449528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:26:58.449528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 09:26:58.449528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:26:58.449528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:26:58.449528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:26:58.449528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:26:58.449528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:26:58.449528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:26:58.449528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:26:58.449528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:26:58.449528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:26:58.449528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:26:58.449528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:26:58.449528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:26:58.449528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:26:58.450482 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:26:58.450482 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:26:58.450482 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 09:26:58.450482 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 09:26:58.450482 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 09:26:58.450482 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 09:26:58.450482 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 09:26:58.450482 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 09:26:58.450482 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 27 09:26:58.450482 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:26:58.450482 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 466/917] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:26:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 09:26:58.456342 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397dfeb0-79ba-4eea-9254-3f504bb2c4b4 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:26:58.582032 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397dfeb0-79ba-4eea-9254-3f504bb2c4b4 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:26:58.630603 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb66c6-0958-4f9b-ad85-3d72a9a0648b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:26:58.631379 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb66c6-0958-4f9b-ad85-3d72a9a0648b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:26:58.631654 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb66c6-0958-4f9b-ad85-3d72a9a0648b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:26:58.631922 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb66c6-0958-4f9b-ad85-3d72a9a0648b None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:26:58.632692 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eb66c6-0958-4f9b-ad85-3d72a9a0648b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:26:58.633000 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eb66c6-0958-4f9b-ad85-3d72a9a0648b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:26:58.634459 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eb66c6-0958-4f9b-ad85-3d72a9a0648b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:26:58.653320 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 452/918] 10.222.0.70 () {56 vars in 1143 bytes} [Thu Oct 27 09:26:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:26:58.659333 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f02dd6-ffdd-4417-9788-5cc4e45ec138 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:26:58.778914 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f02dd6-ffdd-4417-9788-5cc4e45ec138 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:26:58.816173 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207fd1d0-c63b-4d00-9ede-0f3be201f901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:26:58.816855 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207fd1d0-c63b-4d00-9ede-0f3be201f901 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:26:58.817109 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207fd1d0-c63b-4d00-9ede-0f3be201f901 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:26:58.817378 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207fd1d0-c63b-4d00-9ede-0f3be201f901 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:26:58.828096 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207fd1d0-c63b-4d00-9ede-0f3be201f901 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:26:58.830072 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207fd1d0-c63b-4d00-9ede-0f3be201f901 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:26:58.844650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-207fd1d0-c63b-4d00-9ede-0f3be201f901 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:26:58.844650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:26:58.844650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:26:58.844650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:26:58.844650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:26:58.844650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:26:58.844650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:26:58.844650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:26:58.844650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:26:58.844650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:26:58.844650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:26:58.844650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:26:58.844650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:26:58.844650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:26:58.844650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:26:58.844650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:26:58.846219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:26:58.846219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:26:58.846219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:26:58.846219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:26:58.846219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:26:58.846219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:26:58.846219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:26:58.846219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:26:58.846219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:26:58.846219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:26:58.846219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:26:58.846219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:26:58.846219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:26:58.846219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:26:58.846219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:26:58.846219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:26:58.846219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:26:58.847872 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:26:58.847872 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:26:58.847872 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:26:58.847872 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 467/919] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:26:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:26:58.852078 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb66c6-0958-4f9b-ad85-3d72a9a0648b None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:26:58.873189 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:26:58.969081 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb66c6-0958-4f9b-ad85-3d72a9a0648b None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:26:59.005626 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bf574d-b83c-4d00-b1c3-aa457b446a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:26:59.006228 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bf574d-b83c-4d00-b1c3-aa457b446a2b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:26:59.006501 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bf574d-b83c-4d00-b1c3-aa457b446a2b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:26:59.006771 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bf574d-b83c-4d00-b1c3-aa457b446a2b None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:26:59.007393 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bf574d-b83c-4d00-b1c3-aa457b446a2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:26:59.007687 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bf574d-b83c-4d00-b1c3-aa457b446a2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:26:59.009078 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bf574d-b83c-4d00-b1c3-aa457b446a2b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:26:59.020915 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 453/920] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:26:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:26:59.026861 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207fd1d0-c63b-4d00-9ede-0f3be201f901 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:26:59.043987 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:26:59.141129 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207fd1d0-c63b-4d00-9ede-0f3be201f901 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:26:59.186885 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48846005-3047-4d32-8e76-ff9f2d85f4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:26:59.187507 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48846005-3047-4d32-8e76-ff9f2d85f4c8 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:26:59.187782 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48846005-3047-4d32-8e76-ff9f2d85f4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:26:59.188045 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48846005-3047-4d32-8e76-ff9f2d85f4c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:26:59.195809 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48846005-3047-4d32-8e76-ff9f2d85f4c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:26:59.197281 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48846005-3047-4d32-8e76-ff9f2d85f4c8 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:26:59.209830 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-48846005-3047-4d32-8e76-ff9f2d85f4c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:26:59.209830 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:26:59.209830 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:26:59.209830 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:26:59.209830 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:26:59.209830 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:26:59.209830 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:26:59.209830 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:26:59.209830 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:26:59.209830 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:26:59.209830 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:26:59.209830 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:26:59.209830 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 09:26:59.209830 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 09:26:59.209830 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 09:26:59.210760 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 09:26:59.210760 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:26:59.210760 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:26:59.210760 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:26:59.210760 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:26:59.210760 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:26:59.210760 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:26:59.210760 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:26:59.210760 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:26:59.210760 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:26:59.210760 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:26:59.210760 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:26:59.210760 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:26:59.210760 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:26:59.210760 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:26:59.210760 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:26:59.210760 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:26:59.211661 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 09:26:59.211661 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 09:26:59.211661 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 09:26:59.211661 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 09:26:59.211661 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 09:26:59.211661 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 09:26:59.211661 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:26:59.211661 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:26:59.216453 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 468/921] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Oct 27 09:26:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:26:59.217770 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bf574d-b83c-4d00-b1c3-aa457b446a2b None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:26:59.327681 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:26:59.354420 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bf574d-b83c-4d00-b1c3-aa457b446a2b None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:26:59.399143 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c16bb14-6a69-48fe-b6e0-11007d4c5679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:26:59.399811 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16bb14-6a69-48fe-b6e0-11007d4c5679 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:26:59.400091 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16bb14-6a69-48fe-b6e0-11007d4c5679 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:26:59.400377 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16bb14-6a69-48fe-b6e0-11007d4c5679 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:26:59.401071 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c16bb14-6a69-48fe-b6e0-11007d4c5679 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:26:59.401369 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c16bb14-6a69-48fe-b6e0-11007d4c5679 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:26:59.403059 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c16bb14-6a69-48fe-b6e0-11007d4c5679 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:26:59.416212 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 454/922] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Oct 27 09:26:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:26:59.422125 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48846005-3047-4d32-8e76-ff9f2d85f4c8 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:26:59.512308 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:26:59.543937 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48846005-3047-4d32-8e76-ff9f2d85f4c8 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:26:59.582751 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366fbc99-357e-4a87-aa73-f5d206358b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:26:59.584487 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366fbc99-357e-4a87-aa73-f5d206358b98 None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:26:59.584487 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366fbc99-357e-4a87-aa73-f5d206358b98 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:26:59.584487 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366fbc99-357e-4a87-aa73-f5d206358b98 None admin] PATH_INFO: `/v3/projects/16ac205ec6f241d69d5e6b8408f09f38/users/2316b30a6e7b4175bd7b90583584eab5/roles/5146b19817b54113be3f81111ef9d787` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:26:59.605440 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366fbc99-357e-4a87-aa73-f5d206358b98 None admin] RBAC: Authorizing `identity:create_grant(project_id=16ac205ec6f241d69d5e6b8408f09f38, user_id=2316b30a6e7b4175bd7b90583584eab5, role_id=5146b19817b54113be3f81111ef9d787)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:26:59.607089 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366fbc99-357e-4a87-aa73-f5d206358b98 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:26:59.623623 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 469/923] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Oct 27 09:26:59 2022] PUT /identity/v3/projects/16ac205ec6f241d69d5e6b8408f09f38/users/2316b30a6e7b4175bd7b90583584eab5/roles/5146b19817b54113be3f81111ef9d787 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:27:01.979205 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14162a5f-b650-46be-a329-5cb768595776 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:01.979774 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14162a5f-b650-46be-a329-5cb768595776 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:01.979774 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14162a5f-b650-46be-a329-5cb768595776 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:01.981264 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 455/924] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:27:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:27:01.987172 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c531f7-32a6-4d2a-b9ac-7b9ecd65cf51 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:01.987616 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c531f7-32a6-4d2a-b9ac-7b9ecd65cf51 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:01.987729 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c531f7-32a6-4d2a-b9ac-7b9ecd65cf51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:02.005380 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-66c531f7-32a6-4d2a-b9ac-7b9ecd65cf51 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:27:02.006552 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-66c531f7-32a6-4d2a-b9ac-7b9ecd65cf51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:27:02.073587 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 470/925] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:27:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 09:27:02.361307 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14162a5f-b650-46be-a329-5cb768595776 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:27:02.466470 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14162a5f-b650-46be-a329-5cb768595776 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:27:02.503312 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff973d-80c4-49da-80fc-7606b2e8cae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:27:02.503945 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff973d-80c4-49da-80fc-7606b2e8cae5 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:02.504200 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff973d-80c4-49da-80fc-7606b2e8cae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:02.504461 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff973d-80c4-49da-80fc-7606b2e8cae5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:02.511063 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:27:02.515223 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff973d-80c4-49da-80fc-7606b2e8cae5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:27:02.520764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-f5ff973d-80c4-49da-80fc-7606b2e8cae5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 27 09:27:02.520764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:27:02.520764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:27:02.520764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:27:02.520764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:27:02.520764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:27:02.520764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:27:02.520764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:27:02.520764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:27:02.520764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:27:02.520764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:27:02.520764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:27:02.520764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:27:02.520764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:27:02.520764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:27:02.520764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:27:02.521717 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:27:02.521717 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:27:02.521717 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:27:02.521717 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:27:02.521717 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:27:02.521717 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:27:02.521717 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:27:02.521717 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:27:02.521717 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:27:02.521717 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:27:02.521717 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:27:02.521717 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:27:02.521717 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:27:02.521717 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:27:02.521717 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:27:02.521717 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:27:02.522550 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:27:02.522550 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:27:02.522550 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:27:02.522550 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:27:02.522550 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:27:02.522550 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:27:02.522550 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:27:02.522550 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:27:02.522550 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:27:02.522550 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:27:02.522550 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 27 09:27:02.522550 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:27:02.522550 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 456/926] 10.222.0.70 () {56 vars in 1102 bytes} [Thu Oct 27 09:27:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:27:02.527910 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c531f7-32a6-4d2a-b9ac-7b9ecd65cf51 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:27:02.587783 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:27:02.635123 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c531f7-32a6-4d2a-b9ac-7b9ecd65cf51 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:27:02.672254 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8f892-1000-4285-ac04-5f94198037bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:27:02.672923 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8f892-1000-4285-ac04-5f94198037bc None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:02.673200 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8f892-1000-4285-ac04-5f94198037bc None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:02.673487 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8f892-1000-4285-ac04-5f94198037bc None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:02.674139 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a8f892-1000-4285-ac04-5f94198037bc None admin] RBAC: Adding query filter params (name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:27:02.674443 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a8f892-1000-4285-ac04-5f94198037bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:27:02.675789 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a8f892-1000-4285-ac04-5f94198037bc None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:27:02.684279 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 471/927] 10.222.0.70 () {56 vars in 1103 bytes} [Thu Oct 27 09:27:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:27:02.690840 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff973d-80c4-49da-80fc-7606b2e8cae5 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:27:02.796877 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff973d-80c4-49da-80fc-7606b2e8cae5 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:27:02.835151 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff1866b-04b2-4eff-ba9d-36b53d079eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:27:02.835779 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1866b-04b2-4eff-ba9d-36b53d079eda None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:02.836079 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1866b-04b2-4eff-ba9d-36b53d079eda None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:02.836350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1866b-04b2-4eff-ba9d-36b53d079eda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:02.847615 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff1866b-04b2-4eff-ba9d-36b53d079eda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:27:02.849046 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff1866b-04b2-4eff-ba9d-36b53d079eda None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:27:02.864976 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-3ff1866b-04b2-4eff-ba9d-36b53d079eda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:27:02.864976 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:27:02.864976 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:27:02.864976 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:27:02.864976 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:27:02.864976 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:27:02.864976 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:27:02.864976 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:27:02.864976 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:27:02.864976 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:27:02.864976 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:27:02.864976 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:27:02.864976 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:27:02.864976 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:27:02.864976 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:27:02.864976 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:27:02.866539 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:27:02.866539 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:27:02.866539 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:27:02.866539 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:27:02.866539 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:27:02.866539 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:27:02.866539 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:27:02.866539 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:27:02.866539 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:27:02.866539 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:27:02.866539 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:27:02.866539 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:27:02.866539 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:27:02.866539 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:27:02.866539 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:27:02.866539 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:27:02.866539 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:27:02.867799 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:27:02.867799 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:27:02.867799 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:27:02.867799 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 457/928] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:27:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 09:27:02.872129 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8f892-1000-4285-ac04-5f94198037bc None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:27:02.990339 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8f892-1000-4285-ac04-5f94198037bc None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:27:03.009376 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:27:03.029095 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01a4965-9c6a-4274-bc46-68408c985d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:27:03.029699 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a4965-9c6a-4274-bc46-68408c985d3a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:03.029973 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a4965-9c6a-4274-bc46-68408c985d3a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:03.030246 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a4965-9c6a-4274-bc46-68408c985d3a None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:03.030871 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01a4965-9c6a-4274-bc46-68408c985d3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:27:03.031175 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01a4965-9c6a-4274-bc46-68408c985d3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:27:03.032603 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01a4965-9c6a-4274-bc46-68408c985d3a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:27:03.044774 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 472/929] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:27:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:27:03.051244 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff1866b-04b2-4eff-ba9d-36b53d079eda None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:27:03.106955 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:27:03.164144 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff1866b-04b2-4eff-ba9d-36b53d079eda None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:27:03.203600 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84aa15c-3704-4a86-8bed-fceefa3f9dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:27:03.204618 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84aa15c-3704-4a86-8bed-fceefa3f9dd9 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:03.204935 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84aa15c-3704-4a86-8bed-fceefa3f9dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:03.205385 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84aa15c-3704-4a86-8bed-fceefa3f9dd9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:03.213958 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84aa15c-3704-4a86-8bed-fceefa3f9dd9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:27:03.215532 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84aa15c-3704-4a86-8bed-fceefa3f9dd9 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:27:03.228621 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-e84aa15c-3704-4a86-8bed-fceefa3f9dd9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 27 09:27:03.228621 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:27:03.228621 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:27:03.228621 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:27:03.228621 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:27:03.228621 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:27:03.228621 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:27:03.228621 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:27:03.228621 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:27:03.228621 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:27:03.228621 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:27:03.228621 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:27:03.228621 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 09:27:03.228621 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 09:27:03.228621 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 09:27:03.228621 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 09:27:03.229632 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:27:03.229632 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:27:03.229632 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 09:27:03.229632 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:27:03.229632 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 09:27:03.229632 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:27:03.229632 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:27:03.229632 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:27:03.229632 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:27:03.229632 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:27:03.229632 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:27:03.229632 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:27:03.229632 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:27:03.229632 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:27:03.229632 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:27:03.229632 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:27:03.229632 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:27:03.229632 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:27:03.230577 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:27:03.230577 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:27:03.230577 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 09:27:03.230577 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 09:27:03.230577 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 09:27:03.230577 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 09:27:03.230577 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 09:27:03.230577 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 09:27:03.230577 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 27 09:27:03.230577 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:27:03.230577 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 458/930] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:27:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:27:03.236409 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01a4965-9c6a-4274-bc46-68408c985d3a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:27:03.353711 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01a4965-9c6a-4274-bc46-68408c985d3a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:27:03.388453 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589c188a-0d5c-4f51-bec9-9b0cdcee1de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:27:03.389075 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c188a-0d5c-4f51-bec9-9b0cdcee1de9 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:03.389349 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c188a-0d5c-4f51-bec9-9b0cdcee1de9 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:03.389613 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c188a-0d5c-4f51-bec9-9b0cdcee1de9 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:03.390320 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589c188a-0d5c-4f51-bec9-9b0cdcee1de9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:27:03.390624 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589c188a-0d5c-4f51-bec9-9b0cdcee1de9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:27:03.392595 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589c188a-0d5c-4f51-bec9-9b0cdcee1de9 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:27:03.410984 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 473/931] 10.222.0.70 () {56 vars in 1143 bytes} [Thu Oct 27 09:27:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:27:03.415793 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84aa15c-3704-4a86-8bed-fceefa3f9dd9 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:27:03.522300 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84aa15c-3704-4a86-8bed-fceefa3f9dd9 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:27:03.532612 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:27:03.558262 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a9331-ac16-4d43-99be-e26fc153bafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:27:03.559258 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a9331-ac16-4d43-99be-e26fc153bafa None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:03.559604 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a9331-ac16-4d43-99be-e26fc153bafa None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:03.559912 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a9331-ac16-4d43-99be-e26fc153bafa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:03.636147 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5a9331-ac16-4d43-99be-e26fc153bafa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:27:03.638146 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5a9331-ac16-4d43-99be-e26fc153bafa None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:27:03.752841 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 459/932] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Oct 27 09:27:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 27 09:27:03.758533 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589c188a-0d5c-4f51-bec9-9b0cdcee1de9 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:27:03.799416 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:27:03.876038 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589c188a-0d5c-4f51-bec9-9b0cdcee1de9 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:27:03.916826 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6449f316-c968-4474-b359-edb43b177b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:27:03.917565 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449f316-c968-4474-b359-edb43b177b20 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:03.917861 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449f316-c968-4474-b359-edb43b177b20 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:03.918202 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449f316-c968-4474-b359-edb43b177b20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:03.931752 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6449f316-c968-4474-b359-edb43b177b20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:27:03.933602 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6449f316-c968-4474-b359-edb43b177b20 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:27:03.950131 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-6449f316-c968-4474-b359-edb43b177b20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:27:03.950131 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:27:03.950131 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:27:03.950131 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:27:03.950131 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:27:03.950131 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:27:03.950131 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:27:03.950131 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:27:03.950131 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:27:03.950131 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:27:03.950131 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:27:03.950131 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:27:03.950131 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:27:03.950131 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:27:03.950131 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:27:03.950131 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:27:03.952203 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:27:03.952203 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:27:03.952203 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:27:03.952203 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:27:03.952203 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:27:03.952203 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:27:03.952203 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:27:03.952203 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:27:03.952203 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:27:03.952203 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:27:03.952203 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:27:03.952203 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:27:03.952203 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:27:03.952203 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:27:03.952203 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:27:03.952203 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:27:03.952203 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:27:03.954766 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:27:03.954766 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:27:03.954766 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:27:03.954766 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 474/933] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:27:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:27:03.958397 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a9331-ac16-4d43-99be-e26fc153bafa None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:27:03.974404 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:27:04.081184 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a9331-ac16-4d43-99be-e26fc153bafa None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:27:04.119471 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3577eb1b-5e17-4871-88d6-1ca20fa33e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:27:04.120075 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3577eb1b-5e17-4871-88d6-1ca20fa33e3e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:04.120349 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3577eb1b-5e17-4871-88d6-1ca20fa33e3e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:04.120651 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3577eb1b-5e17-4871-88d6-1ca20fa33e3e None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:04.121280 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3577eb1b-5e17-4871-88d6-1ca20fa33e3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:27:04.121585 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3577eb1b-5e17-4871-88d6-1ca20fa33e3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:27:04.122941 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3577eb1b-5e17-4871-88d6-1ca20fa33e3e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:27:04.133924 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 460/934] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:27:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:27:04.139528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6449f316-c968-4474-b359-edb43b177b20 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:27:04.236966 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:27:04.255010 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6449f316-c968-4474-b359-edb43b177b20 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:27:04.298587 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c303c93-6760-4b21-be05-e079fc2f5a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:27:04.299535 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c303c93-6760-4b21-be05-e079fc2f5a20 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:04.299831 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c303c93-6760-4b21-be05-e079fc2f5a20 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:04.300247 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c303c93-6760-4b21-be05-e079fc2f5a20 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:04.308148 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c303c93-6760-4b21-be05-e079fc2f5a20 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:27:04.309648 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c303c93-6760-4b21-be05-e079fc2f5a20 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:27:04.321472 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-5c303c93-6760-4b21-be05-e079fc2f5a20 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:27:04.321472 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:27:04.321472 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:27:04.321472 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:27:04.321472 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:27:04.321472 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:27:04.321472 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:27:04.321472 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:27:04.321472 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:27:04.321472 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:27:04.321472 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:27:04.321472 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:27:04.321472 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 09:27:04.321472 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 09:27:04.321472 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 09:27:04.322528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 09:27:04.322528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:27:04.322528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:27:04.322528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:27:04.322528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:27:04.322528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:27:04.322528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:27:04.322528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:27:04.322528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:27:04.322528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:27:04.322528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:27:04.322528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:27:04.322528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:27:04.322528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:27:04.322528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:27:04.322528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:27:04.322528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:27:04.323458 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 09:27:04.323458 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 09:27:04.323458 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 09:27:04.323458 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 09:27:04.323458 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 09:27:04.323458 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 09:27:04.323458 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:27:04.323458 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:27:04.323458 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 475/935] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Oct 27 09:27:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:27:04.328576 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3577eb1b-5e17-4871-88d6-1ca20fa33e3e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:27:04.408285 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:27:04.445383 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3577eb1b-5e17-4871-88d6-1ca20fa33e3e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:27:04.481384 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2484b7db-85b9-4a88-ad71-746b548201b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:27:04.482167 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2484b7db-85b9-4a88-ad71-746b548201b4 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:04.482482 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2484b7db-85b9-4a88-ad71-746b548201b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:04.482910 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2484b7db-85b9-4a88-ad71-746b548201b4 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:04.483785 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2484b7db-85b9-4a88-ad71-746b548201b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:27:04.484115 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2484b7db-85b9-4a88-ad71-746b548201b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:27:04.485626 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2484b7db-85b9-4a88-ad71-746b548201b4 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:27:04.497929 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 461/936] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Oct 27 09:27:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:27:04.504967 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c303c93-6760-4b21-be05-e079fc2f5a20 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:27:04.609733 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c303c93-6760-4b21-be05-e079fc2f5a20 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:27:04.649557 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eb1c8d-2a2d-409c-8087-771c759b9991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:27:04.650191 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb1c8d-2a2d-409c-8087-771c759b9991 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:04.650485 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb1c8d-2a2d-409c-8087-771c759b9991 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:04.650758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb1c8d-2a2d-409c-8087-771c759b9991 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:04.651602 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eb1c8d-2a2d-409c-8087-771c759b9991 None admin] RBAC: Adding query filter params (role.id=5146b19817b54113be3f81111ef9d787, scope.project.id=16ac205ec6f241d69d5e6b8408f09f38, user.id=2316b30a6e7b4175bd7b90583584eab5) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:27:04.651887 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eb1c8d-2a2d-409c-8087-771c759b9991 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:27:04.653459 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eb1c8d-2a2d-409c-8087-771c759b9991 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:27:04.659940 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 476/937] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Oct 27 09:27:04 2022] GET /identity/v3/role_assignments?user.id=2316b30a6e7b4175bd7b90583584eab5&scope.project.id=16ac205ec6f241d69d5e6b8408f09f38&role.id=5146b19817b54113be3f81111ef9d787 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:27:06.852356 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538bba24-08b0-4179-a26f-a3b0683c03a7 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:06.852946 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538bba24-08b0-4179-a26f-a3b0683c03a7 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:06.852946 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538bba24-08b0-4179-a26f-a3b0683c03a7 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:06.854273 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 462/938] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:27:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 27 09:27:06.860085 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af304ff-1f76-4f58-966a-50a408b929d6 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:06.860536 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af304ff-1f76-4f58-966a-50a408b929d6 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:06.860677 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af304ff-1f76-4f58-966a-50a408b929d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:06.883133 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-4af304ff-1f76-4f58-966a-50a408b929d6 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:27:06.884395 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4af304ff-1f76-4f58-966a-50a408b929d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:27:06.887530 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:27:06.959563 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 477/939] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:27:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 09:27:07.228603 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538bba24-08b0-4179-a26f-a3b0683c03a7 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:27:07.347794 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538bba24-08b0-4179-a26f-a3b0683c03a7 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:27:07.380168 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:27:07.390312 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910eca39-413c-4c55-a84f-56e630ec7089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:27:07.390928 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910eca39-413c-4c55-a84f-56e630ec7089 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:07.391200 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910eca39-413c-4c55-a84f-56e630ec7089 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:07.391490 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910eca39-413c-4c55-a84f-56e630ec7089 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:07.402623 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910eca39-413c-4c55-a84f-56e630ec7089 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:27:07.408371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-910eca39-413c-4c55-a84f-56e630ec7089 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 09:27:07.408371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:27:07.408371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:27:07.408371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:27:07.408371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:27:07.408371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:27:07.408371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:27:07.408371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:27:07.408371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:27:07.408371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:27:07.408371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:27:07.408371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:27:07.408371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:27:07.408371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:27:07.408371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:27:07.408371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:27:07.409730 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:27:07.409730 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:27:07.409730 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:27:07.409730 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:27:07.409730 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:27:07.409730 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:27:07.409730 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:27:07.409730 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:27:07.409730 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:27:07.409730 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:27:07.409730 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:27:07.409730 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:27:07.409730 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:27:07.409730 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:27:07.409730 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:27:07.409730 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:27:07.411060 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:27:07.411060 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:27:07.411060 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:27:07.411060 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:27:07.411060 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:27:07.411060 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:27:07.411060 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:27:07.411060 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:27:07.411060 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:27:07.411060 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:27:07.411060 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 09:27:07.411060 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:27:07.411060 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 463/940] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Oct 27 09:27:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:27:07.416121 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af304ff-1f76-4f58-966a-50a408b929d6 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:27:07.526839 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af304ff-1f76-4f58-966a-50a408b929d6 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:27:07.562193 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1968e3-c8b1-4928-9e20-7c885b59a6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:27:07.563040 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1968e3-c8b1-4928-9e20-7c885b59a6f2 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:07.563923 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1968e3-c8b1-4928-9e20-7c885b59a6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:07.564332 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1968e3-c8b1-4928-9e20-7c885b59a6f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:07.565101 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1968e3-c8b1-4928-9e20-7c885b59a6f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:27:07.565598 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1968e3-c8b1-4928-9e20-7c885b59a6f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:27:07.567171 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1968e3-c8b1-4928-9e20-7c885b59a6f2 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:27:07.577168 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 478/941] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Oct 27 09:27:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:27:07.583655 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910eca39-413c-4c55-a84f-56e630ec7089 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:27:07.703620 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910eca39-413c-4c55-a84f-56e630ec7089 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:27:07.741075 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3164b9-ac64-4142-b09c-f02dcecd00c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:27:07.741703 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3164b9-ac64-4142-b09c-f02dcecd00c8 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:07.741996 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3164b9-ac64-4142-b09c-f02dcecd00c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:07.742252 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3164b9-ac64-4142-b09c-f02dcecd00c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:07.755545 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3164b9-ac64-4142-b09c-f02dcecd00c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:27:07.757035 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3164b9-ac64-4142-b09c-f02dcecd00c8 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:27:07.774070 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-dd3164b9-ac64-4142-b09c-f02dcecd00c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:27:07.774070 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:27:07.774070 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:27:07.774070 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:27:07.774070 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:27:07.774070 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:27:07.774070 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:27:07.774070 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:27:07.774070 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:27:07.774070 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:27:07.774070 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:27:07.774070 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:27:07.774070 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:27:07.774070 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:27:07.774070 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:27:07.774070 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:27:07.775483 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:27:07.775483 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:27:07.775483 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:27:07.775483 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:27:07.775483 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:27:07.775483 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:27:07.775483 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:27:07.775483 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:27:07.775483 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:27:07.775483 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:27:07.775483 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:27:07.775483 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:27:07.775483 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:27:07.775483 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:27:07.775483 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:27:07.775483 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:27:07.775483 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:27:07.776830 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:27:07.776830 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:27:07.776830 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:27:07.776830 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 464/942] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:27:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:27:07.781488 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1968e3-c8b1-4928-9e20-7c885b59a6f2 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:27:07.784319 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:27:07.898599 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1968e3-c8b1-4928-9e20-7c885b59a6f2 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:27:07.937805 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f46d7-edcf-4bd0-b0b7-64f954d1b109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:27:07.938720 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f46d7-edcf-4bd0-b0b7-64f954d1b109 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:07.939233 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f46d7-edcf-4bd0-b0b7-64f954d1b109 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:07.939717 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f46d7-edcf-4bd0-b0b7-64f954d1b109 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:07.940739 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428f46d7-edcf-4bd0-b0b7-64f954d1b109 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:27:07.941303 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428f46d7-edcf-4bd0-b0b7-64f954d1b109 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:27:07.943231 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428f46d7-edcf-4bd0-b0b7-64f954d1b109 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:27:07.964491 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 479/943] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:27:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:27:07.966020 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3164b9-ac64-4142-b09c-f02dcecd00c8 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:27:08.017696 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:27:08.081301 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3164b9-ac64-4142-b09c-f02dcecd00c8 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:27:08.122360 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e5c64f-5be7-49bc-86ea-aac3618ea111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:27:08.123007 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5c64f-5be7-49bc-86ea-aac3618ea111 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:08.123286 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5c64f-5be7-49bc-86ea-aac3618ea111 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:08.123553 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5c64f-5be7-49bc-86ea-aac3618ea111 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:08.131075 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e5c64f-5be7-49bc-86ea-aac3618ea111 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:27:08.132499 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e5c64f-5be7-49bc-86ea-aac3618ea111 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:27:08.143430 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-78e5c64f-5be7-49bc-86ea-aac3618ea111 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 27 09:27:08.143430 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:27:08.143430 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:27:08.143430 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:27:08.143430 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:27:08.143430 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:27:08.143430 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:27:08.143430 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:27:08.143430 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:27:08.143430 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:27:08.143430 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:27:08.143430 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:27:08.143430 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 09:27:08.143430 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 09:27:08.143430 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 09:27:08.143430 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 09:27:08.144586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:27:08.144586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:27:08.144586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 09:27:08.144586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:27:08.144586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 09:27:08.144586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:27:08.144586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:27:08.144586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:27:08.144586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:27:08.144586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:27:08.144586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:27:08.144586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:27:08.144586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:27:08.144586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:27:08.144586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:27:08.144586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:27:08.144586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:27:08.144586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:27:08.145759 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:27:08.145759 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:27:08.145759 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 09:27:08.145759 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 09:27:08.145759 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 09:27:08.145759 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 09:27:08.145759 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 09:27:08.145759 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 09:27:08.145759 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 27 09:27:08.145759 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:27:08.145759 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 465/944] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:27:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:27:08.150278 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f46d7-edcf-4bd0-b0b7-64f954d1b109 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:27:08.224238 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:27:08.274341 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f46d7-edcf-4bd0-b0b7-64f954d1b109 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:27:08.311518 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8e9362-db88-4c91-8eac-0a210bf2118a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:27:08.312120 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e9362-db88-4c91-8eac-0a210bf2118a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:08.312392 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e9362-db88-4c91-8eac-0a210bf2118a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:08.312718 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e9362-db88-4c91-8eac-0a210bf2118a None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:08.313487 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8e9362-db88-4c91-8eac-0a210bf2118a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:27:08.313803 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8e9362-db88-4c91-8eac-0a210bf2118a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:27:08.315375 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8e9362-db88-4c91-8eac-0a210bf2118a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:27:08.334673 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 480/945] 10.222.0.70 () {56 vars in 1143 bytes} [Thu Oct 27 09:27:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:27:08.340399 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e5c64f-5be7-49bc-86ea-aac3618ea111 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:27:08.464531 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e5c64f-5be7-49bc-86ea-aac3618ea111 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:27:08.466205 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:27:08.505615 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61461f2f-b3b7-4ba3-9ec1-6611e13388b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:27:08.506463 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61461f2f-b3b7-4ba3-9ec1-6611e13388b0 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:08.506813 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61461f2f-b3b7-4ba3-9ec1-6611e13388b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:08.507080 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61461f2f-b3b7-4ba3-9ec1-6611e13388b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:08.521169 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61461f2f-b3b7-4ba3-9ec1-6611e13388b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:27:08.522942 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61461f2f-b3b7-4ba3-9ec1-6611e13388b0 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:27:08.540663 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-61461f2f-b3b7-4ba3-9ec1-6611e13388b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:27:08.540663 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:27:08.540663 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:27:08.540663 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:27:08.540663 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:27:08.540663 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:27:08.540663 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:27:08.540663 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:27:08.540663 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:27:08.540663 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:27:08.540663 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:27:08.540663 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:27:08.540663 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:27:08.540663 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:27:08.540663 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:27:08.540663 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:27:08.541648 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:27:08.541648 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:27:08.541648 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:27:08.541648 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:27:08.541648 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:27:08.541648 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:27:08.541648 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:27:08.541648 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:27:08.541648 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:27:08.541648 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:27:08.541648 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:27:08.541648 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:27:08.541648 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:27:08.541648 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:27:08.541648 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:27:08.541648 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:27:08.541648 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:27:08.542522 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:27:08.542522 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:27:08.542522 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:27:08.542522 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 466/946] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:27:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:27:08.548423 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8e9362-db88-4c91-8eac-0a210bf2118a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:27:08.665201 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8e9362-db88-4c91-8eac-0a210bf2118a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:27:08.689461 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:27:08.707010 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779e722-c516-4478-bcba-877f2aa870e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:27:08.707742 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779e722-c516-4478-bcba-877f2aa870e4 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:08.708052 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779e722-c516-4478-bcba-877f2aa870e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:08.708610 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779e722-c516-4478-bcba-877f2aa870e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:08.709320 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3779e722-c516-4478-bcba-877f2aa870e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:27:08.709656 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3779e722-c516-4478-bcba-877f2aa870e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:27:08.711348 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3779e722-c516-4478-bcba-877f2aa870e4 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:27:08.724959 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 481/947] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:27:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:27:08.730996 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61461f2f-b3b7-4ba3-9ec1-6611e13388b0 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:27:08.845839 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61461f2f-b3b7-4ba3-9ec1-6611e13388b0 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:27:08.880404 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2d61c8-7144-4c7d-810c-332c25c0b5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:27:08.881068 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d61c8-7144-4c7d-810c-332c25c0b5aa None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:08.881348 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d61c8-7144-4c7d-810c-332c25c0b5aa None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:08.881620 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d61c8-7144-4c7d-810c-332c25c0b5aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:08.888186 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2d61c8-7144-4c7d-810c-332c25c0b5aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:27:08.889666 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2d61c8-7144-4c7d-810c-332c25c0b5aa None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:27:08.903026 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-3f2d61c8-7144-4c7d-810c-332c25c0b5aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:27:08.903026 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:27:08.903026 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:27:08.903026 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:27:08.903026 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:27:08.903026 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:27:08.903026 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:27:08.903026 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:27:08.903026 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:27:08.903026 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:27:08.903026 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:27:08.903026 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:27:08.903026 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 09:27:08.903026 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 09:27:08.903026 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 09:27:08.904182 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 09:27:08.904182 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:27:08.904182 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:27:08.904182 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:27:08.904182 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:27:08.904182 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:27:08.904182 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:27:08.904182 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:27:08.904182 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:27:08.904182 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:27:08.904182 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:27:08.904182 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:27:08.904182 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:27:08.904182 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:27:08.904182 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:27:08.904182 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:27:08.904182 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:27:08.905894 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 09:27:08.905894 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 09:27:08.905894 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 09:27:08.905894 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 09:27:08.905894 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 09:27:08.905894 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 09:27:08.905894 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:27:08.905894 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:27:08.907056 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 467/948] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Oct 27 09:27:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:27:08.912928 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779e722-c516-4478-bcba-877f2aa870e4 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:27:09.034986 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779e722-c516-4478-bcba-877f2aa870e4 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:27:09.071082 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a680558b-edb8-4b0a-808a-283af8601b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:27:09.071873 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680558b-edb8-4b0a-808a-283af8601b01 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:09.072198 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680558b-edb8-4b0a-808a-283af8601b01 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:09.072692 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680558b-edb8-4b0a-808a-283af8601b01 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:09.073387 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a680558b-edb8-4b0a-808a-283af8601b01 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:27:09.073885 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a680558b-edb8-4b0a-808a-283af8601b01 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:27:09.075459 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a680558b-edb8-4b0a-808a-283af8601b01 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:27:09.093242 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 482/949] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Oct 27 09:27:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:27:09.099537 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2d61c8-7144-4c7d-810c-332c25c0b5aa None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:27:09.120459 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:27:09.214239 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2d61c8-7144-4c7d-810c-332c25c0b5aa None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:27:09.252319 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b76bbbc-ab06-441b-ae26-3bfb9b6020d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:27:09.253058 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b76bbbc-ab06-441b-ae26-3bfb9b6020d8 None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:09.253339 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b76bbbc-ab06-441b-ae26-3bfb9b6020d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:09.253644 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b76bbbc-ab06-441b-ae26-3bfb9b6020d8 None admin] PATH_INFO: `/v3/projects/16ac205ec6f241d69d5e6b8408f09f38/users/2316b30a6e7b4175bd7b90583584eab5/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:09.255611 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b76bbbc-ab06-441b-ae26-3bfb9b6020d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=16ac205ec6f241d69d5e6b8408f09f38, user_id=2316b30a6e7b4175bd7b90583584eab5, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:27:09.257249 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b76bbbc-ab06-441b-ae26-3bfb9b6020d8 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:27:09.269462 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 468/950] 10.222.0.70 () {58 vars in 1439 bytes} [Thu Oct 27 09:27:09 2022] PUT /identity/v3/projects/16ac205ec6f241d69d5e6b8408f09f38/users/2316b30a6e7b4175bd7b90583584eab5/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:27:11.538635 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb599a-2b10-4d1c-af5a-6b38b0cdf83f None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:11.539174 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb599a-2b10-4d1c-af5a-6b38b0cdf83f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:11.539174 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb599a-2b10-4d1c-af5a-6b38b0cdf83f None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:11.540437 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 483/951] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:27:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:27:11.546357 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c865e98c-e19b-45e3-8127-a1a328e1eaba None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:11.546871 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c865e98c-e19b-45e3-8127-a1a328e1eaba None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:11.547013 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c865e98c-e19b-45e3-8127-a1a328e1eaba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:11.565248 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-c865e98c-e19b-45e3-8127-a1a328e1eaba None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:27:11.566440 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c865e98c-e19b-45e3-8127-a1a328e1eaba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:27:11.642341 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 469/952] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:27:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 09:27:11.923914 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfb599a-2b10-4d1c-af5a-6b38b0cdf83f None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:27:11.981866 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:27:12.056814 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfb599a-2b10-4d1c-af5a-6b38b0cdf83f None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:27:12.095195 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28abc326-4fb2-4d72-96b3-1e107d91de3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:27:12.095812 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28abc326-4fb2-4d72-96b3-1e107d91de3f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:12.096095 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28abc326-4fb2-4d72-96b3-1e107d91de3f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:12.096365 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28abc326-4fb2-4d72-96b3-1e107d91de3f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:12.106688 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28abc326-4fb2-4d72-96b3-1e107d91de3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 09:27:12.111969 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-28abc326-4fb2-4d72-96b3-1e107d91de3f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 09:27:12.111969 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:27:12.111969 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:27:12.111969 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:27:12.111969 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:27:12.111969 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:27:12.111969 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:27:12.111969 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:27:12.111969 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:27:12.111969 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:27:12.111969 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:27:12.111969 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:27:12.111969 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 09:27:12.111969 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 09:27:12.111969 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 09:27:12.111969 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 09:27:12.113213 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 09:27:12.113213 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 09:27:12.113213 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 09:27:12.113213 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 09:27:12.113213 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:27:12.113213 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:27:12.113213 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:27:12.113213 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:27:12.113213 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:27:12.113213 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:27:12.113213 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:27:12.113213 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:27:12.113213 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:27:12.113213 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:27:12.113213 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:27:12.113213 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:27:12.114391 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:27:12.114391 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:27:12.114391 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:27:12.114391 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:27:12.114391 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 09:27:12.114391 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 09:27:12.114391 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 09:27:12.114391 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 09:27:12.114391 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 09:27:12.114391 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 09:27:12.114391 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 09:27:12.114391 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:27:12.114391 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 484/953] 10.222.0.70 () {56 vars in 1096 bytes} [Thu Oct 27 09:27:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 09:27:12.119297 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c865e98c-e19b-45e3-8127-a1a328e1eaba None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:27:12.122462 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:27:12.230609 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c865e98c-e19b-45e3-8127-a1a328e1eaba None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:27:12.265882 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71585c9-6864-4a6c-a729-ed9248ee67a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:27:12.266570 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71585c9-6864-4a6c-a729-ed9248ee67a1 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:12.266852 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71585c9-6864-4a6c-a729-ed9248ee67a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:12.267122 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71585c9-6864-4a6c-a729-ed9248ee67a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:12.267803 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71585c9-6864-4a6c-a729-ed9248ee67a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:27:12.270525 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71585c9-6864-4a6c-a729-ed9248ee67a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:27:12.270525 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71585c9-6864-4a6c-a729-ed9248ee67a1 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:27:12.278318 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 470/954] 10.222.0.70 () {56 vars in 1099 bytes} [Thu Oct 27 09:27:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:27:12.285358 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28abc326-4fb2-4d72-96b3-1e107d91de3f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:27:12.383443 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:27:12.386945 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28abc326-4fb2-4d72-96b3-1e107d91de3f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:27:12.423012 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78d4fcd-ea08-40a6-8043-9028708da70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:27:12.423664 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d4fcd-ea08-40a6-8043-9028708da70d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:12.423938 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d4fcd-ea08-40a6-8043-9028708da70d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:12.424202 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d4fcd-ea08-40a6-8043-9028708da70d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:12.434504 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78d4fcd-ea08-40a6-8043-9028708da70d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:27:12.435967 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78d4fcd-ea08-40a6-8043-9028708da70d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:27:12.451052 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-c78d4fcd-ea08-40a6-8043-9028708da70d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:27:12.451052 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:27:12.451052 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:27:12.451052 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:27:12.451052 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:27:12.451052 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:27:12.451052 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:27:12.451052 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:27:12.451052 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:27:12.451052 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:27:12.451052 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:27:12.451052 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:27:12.451052 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:27:12.451052 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:27:12.451052 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:27:12.451052 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:27:12.452276 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:27:12.452276 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:27:12.452276 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:27:12.452276 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:27:12.452276 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:27:12.452276 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:27:12.452276 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:27:12.452276 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:27:12.452276 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:27:12.452276 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:27:12.452276 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:27:12.452276 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:27:12.452276 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:27:12.452276 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:27:12.452276 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:27:12.452276 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:27:12.452276 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:27:12.453344 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:27:12.453344 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:27:12.453344 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:27:12.453344 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 485/955] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:27:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:27:12.458768 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71585c9-6864-4a6c-a729-ed9248ee67a1 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:27:12.531599 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:27:12.566700 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71585c9-6864-4a6c-a729-ed9248ee67a1 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:27:12.602082 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02735814-359d-4132-8833-d1b697fce87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:27:12.602691 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02735814-359d-4132-8833-d1b697fce87e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:12.602972 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02735814-359d-4132-8833-d1b697fce87e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:12.603240 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02735814-359d-4132-8833-d1b697fce87e None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:12.603870 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02735814-359d-4132-8833-d1b697fce87e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:27:12.604177 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02735814-359d-4132-8833-d1b697fce87e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:27:12.605574 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02735814-359d-4132-8833-d1b697fce87e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:27:12.617458 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 471/956] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:27:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:27:12.623503 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78d4fcd-ea08-40a6-8043-9028708da70d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:27:12.728771 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78d4fcd-ea08-40a6-8043-9028708da70d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:27:12.762495 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9ae38c-82ff-4ed4-b14d-83a8ab7af598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:27:12.763121 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ae38c-82ff-4ed4-b14d-83a8ab7af598 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:12.763445 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ae38c-82ff-4ed4-b14d-83a8ab7af598 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:12.763761 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ae38c-82ff-4ed4-b14d-83a8ab7af598 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:12.772279 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9ae38c-82ff-4ed4-b14d-83a8ab7af598 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:27:12.773749 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9ae38c-82ff-4ed4-b14d-83a8ab7af598 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:27:12.785112 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-4b9ae38c-82ff-4ed4-b14d-83a8ab7af598 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 27 09:27:12.785112 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:27:12.785112 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:27:12.785112 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:27:12.785112 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:27:12.785112 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:27:12.785112 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:27:12.785112 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:27:12.785112 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:27:12.785112 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:27:12.785112 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:27:12.785112 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:27:12.785112 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 09:27:12.785112 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 09:27:12.785112 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 09:27:12.785112 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 09:27:12.786219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:27:12.786219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:27:12.786219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 09:27:12.786219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:27:12.786219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 09:27:12.786219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 09:27:12.786219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:27:12.786219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:27:12.786219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:27:12.786219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:27:12.786219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:27:12.786219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:27:12.786219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:27:12.786219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:27:12.786219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:27:12.786219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:27:12.786219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:27:12.786219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:27:12.787392 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:27:12.787392 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:27:12.787392 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 09:27:12.787392 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 09:27:12.787392 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 09:27:12.787392 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 09:27:12.787392 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 09:27:12.787392 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 09:27:12.787392 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 27 09:27:12.787392 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:27:12.791034 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 486/957] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:27:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 09:27:12.792296 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02735814-359d-4132-8833-d1b697fce87e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:27:12.898647 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02735814-359d-4132-8833-d1b697fce87e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:27:12.927940 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:27:12.940693 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70061203-7aaa-4443-b96b-4ca2537afd5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:27:12.941404 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70061203-7aaa-4443-b96b-4ca2537afd5f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:12.941718 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70061203-7aaa-4443-b96b-4ca2537afd5f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:12.942005 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70061203-7aaa-4443-b96b-4ca2537afd5f None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:12.942961 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70061203-7aaa-4443-b96b-4ca2537afd5f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:27:12.943301 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70061203-7aaa-4443-b96b-4ca2537afd5f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:27:12.945141 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70061203-7aaa-4443-b96b-4ca2537afd5f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:27:12.968552 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 472/958] 10.222.0.70 () {56 vars in 1143 bytes} [Thu Oct 27 09:27:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:27:12.974334 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9ae38c-82ff-4ed4-b14d-83a8ab7af598 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:27:12.994285 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:27:13.095668 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9ae38c-82ff-4ed4-b14d-83a8ab7af598 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:27:13.132840 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2ca16c-1f8e-404a-8b36-af49b4fbfcda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:27:13.133446 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ca16c-1f8e-404a-8b36-af49b4fbfcda None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:13.133730 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ca16c-1f8e-404a-8b36-af49b4fbfcda None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:13.134001 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ca16c-1f8e-404a-8b36-af49b4fbfcda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:13.211192 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2ca16c-1f8e-404a-8b36-af49b4fbfcda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:27:13.212739 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2ca16c-1f8e-404a-8b36-af49b4fbfcda None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:27:13.227208 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:27:13.326550 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 487/959] 10.222.0.70 () {58 vars in 1282 bytes} [Thu Oct 27 09:27:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 27 09:27:13.333411 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70061203-7aaa-4443-b96b-4ca2537afd5f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:27:13.439026 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70061203-7aaa-4443-b96b-4ca2537afd5f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:27:13.475456 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453f958-6560-4228-bbcb-845724a8c778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:27:13.476101 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453f958-6560-4228-bbcb-845724a8c778 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:13.476380 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453f958-6560-4228-bbcb-845724a8c778 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:13.476691 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453f958-6560-4228-bbcb-845724a8c778 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:13.488355 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f453f958-6560-4228-bbcb-845724a8c778 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:27:13.489865 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f453f958-6560-4228-bbcb-845724a8c778 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:27:13.506911 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-f453f958-6560-4228-bbcb-845724a8c778 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:27:13.506911 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:27:13.506911 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:27:13.506911 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:27:13.506911 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:27:13.506911 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:27:13.506911 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:27:13.506911 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:27:13.506911 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:27:13.506911 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:27:13.506911 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:27:13.506911 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:27:13.506911 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 09:27:13.506911 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 09:27:13.506911 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 09:27:13.506911 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 09:27:13.508005 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:27:13.508005 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:27:13.508005 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:27:13.508005 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:27:13.508005 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:27:13.508005 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:27:13.508005 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:27:13.508005 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:27:13.508005 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:27:13.508005 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:27:13.508005 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:27:13.508005 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:27:13.508005 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:27:13.508005 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:27:13.508005 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:27:13.508005 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:27:13.508005 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 09:27:13.508943 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 09:27:13.508943 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 09:27:13.508943 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:27:13.508943 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 473/960] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:27:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:27:13.513989 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2ca16c-1f8e-404a-8b36-af49b4fbfcda None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:27:13.616651 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2ca16c-1f8e-404a-8b36-af49b4fbfcda None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:27:13.630543 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:27:13.653886 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1169d67-5d8d-4865-b41b-f386a4a47599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:27:13.654502 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1169d67-5d8d-4865-b41b-f386a4a47599 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:13.654775 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1169d67-5d8d-4865-b41b-f386a4a47599 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:13.655042 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1169d67-5d8d-4865-b41b-f386a4a47599 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:13.655669 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1169d67-5d8d-4865-b41b-f386a4a47599 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:27:13.655964 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1169d67-5d8d-4865-b41b-f386a4a47599 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:27:13.657921 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1169d67-5d8d-4865-b41b-f386a4a47599 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:27:13.669822 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 488/961] 10.222.0.70 () {56 vars in 1109 bytes} [Thu Oct 27 09:27:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:27:13.675555 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453f958-6560-4228-bbcb-845724a8c778 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:27:13.712166 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:27:13.780034 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453f958-6560-4228-bbcb-845724a8c778 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:27:13.814848 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32660c69-db0b-4d2b-b92a-afb3fd551e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:27:13.815475 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32660c69-db0b-4d2b-b92a-afb3fd551e8a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:13.815753 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32660c69-db0b-4d2b-b92a-afb3fd551e8a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:13.816019 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32660c69-db0b-4d2b-b92a-afb3fd551e8a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:13.822809 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32660c69-db0b-4d2b-b92a-afb3fd551e8a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:27:13.824249 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32660c69-db0b-4d2b-b92a-afb3fd551e8a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:27:13.835915 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-32660c69-db0b-4d2b-b92a-afb3fd551e8a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:27:13.835915 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:27:13.835915 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:27:13.835915 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:27:13.835915 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:27:13.835915 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:27:13.835915 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:27:13.835915 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:27:13.835915 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:27:13.835915 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:27:13.835915 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:27:13.835915 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:27:13.835915 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 09:27:13.835915 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 09:27:13.835915 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 09:27:13.837053 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 09:27:13.837053 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:27:13.837053 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:27:13.837053 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:27:13.837053 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:27:13.837053 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:27:13.837053 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:27:13.837053 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:27:13.837053 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 27 09:27:13.837053 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:27:13.837053 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:27:13.837053 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:27:13.837053 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:27:13.837053 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:27:13.837053 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:27:13.837053 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:27:13.837053 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:27:13.838144 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 09:27:13.838144 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 09:27:13.838144 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 09:27:13.838144 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 09:27:13.838144 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 09:27:13.838144 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 09:27:13.838144 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 09:27:13.838144 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 27 09:27:13.838144 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 474/962] 10.222.0.70 () {56 vars in 1111 bytes} [Thu Oct 27 09:27:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:27:13.843456 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1169d67-5d8d-4865-b41b-f386a4a47599 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:27:13.952320 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1169d67-5d8d-4865-b41b-f386a4a47599 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:27:13.993219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd0036c-b09e-4569-b51c-10ce069bb7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:27:13.993802 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd0036c-b09e-4569-b51c-10ce069bb7d5 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:13.994124 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd0036c-b09e-4569-b51c-10ce069bb7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:13.994402 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd0036c-b09e-4569-b51c-10ce069bb7d5 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:13.995088 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd0036c-b09e-4569-b51c-10ce069bb7d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:27:13.995394 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd0036c-b09e-4569-b51c-10ce069bb7d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:27:13.996862 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd0036c-b09e-4569-b51c-10ce069bb7d5 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:27:14.008778 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 489/963] 10.222.0.70 () {56 vars in 1148 bytes} [Thu Oct 27 09:27:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:27:14.015002 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32660c69-db0b-4d2b-b92a-afb3fd551e8a None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:27:14.106079 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:27:14.112770 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32660c69-db0b-4d2b-b92a-afb3fd551e8a None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:27:14.144503 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7195c0-7b0b-4e05-ba08-2d2d3d56aa15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:27:14.145105 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7195c0-7b0b-4e05-ba08-2d2d3d56aa15 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:14.145381 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7195c0-7b0b-4e05-ba08-2d2d3d56aa15 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:14.145647 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7195c0-7b0b-4e05-ba08-2d2d3d56aa15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:14.146344 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7195c0-7b0b-4e05-ba08-2d2d3d56aa15 None admin] RBAC: Adding query filter params (role.id=0dfd839b7a194aee96f82f4ebc1620a7, scope.project.id=16ac205ec6f241d69d5e6b8408f09f38, user.id=2316b30a6e7b4175bd7b90583584eab5) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:27:14.146644 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7195c0-7b0b-4e05-ba08-2d2d3d56aa15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:27:14.148054 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7195c0-7b0b-4e05-ba08-2d2d3d56aa15 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:27:14.153604 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 475/964] 10.222.0.70 () {56 vars in 1374 bytes} [Thu Oct 27 09:27:14 2022] GET /identity/v3/role_assignments?user.id=2316b30a6e7b4175bd7b90583584eab5&scope.project.id=16ac205ec6f241d69d5e6b8408f09f38&role.id=0dfd839b7a194aee96f82f4ebc1620a7 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:27:16.294303 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e74720-e1a1-4784-956d-b9f4f86b26b7 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:16.294700 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e74720-e1a1-4784-956d-b9f4f86b26b7 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:16.294853 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e74720-e1a1-4784-956d-b9f4f86b26b7 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:16.296196 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 490/965] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:27:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:27:16.301897 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f250d-9aa7-4bbd-802b-2622606ba078 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:16.302302 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f250d-9aa7-4bbd-802b-2622606ba078 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:16.302433 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f250d-9aa7-4bbd-802b-2622606ba078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:16.320176 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-641f250d-9aa7-4bbd-802b-2622606ba078 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:27:16.321388 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-641f250d-9aa7-4bbd-802b-2622606ba078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:27:16.384586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 476/966] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:27:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 09:27:16.652723 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e74720-e1a1-4784-956d-b9f4f86b26b7 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:27:16.682727 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:27:16.750251 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e74720-e1a1-4784-956d-b9f4f86b26b7 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:27:16.782870 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c462f45f-51c6-4b17-a875-70e545d463d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:27:16.783471 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462f45f-51c6-4b17-a875-70e545d463d1 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:16.783745 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462f45f-51c6-4b17-a875-70e545d463d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:16.784008 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462f45f-51c6-4b17-a875-70e545d463d1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:16.784600 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c462f45f-51c6-4b17-a875-70e545d463d1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:27:16.785936 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c462f45f-51c6-4b17-a875-70e545d463d1 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:27:16.794343 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-c462f45f-51c6-4b17-a875-70e545d463d1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 27 09:27:16.794343 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:27:16.794343 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 09:27:16.794343 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 09:27:16.794343 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 09:27:16.794343 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 09:27:16.794343 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 09:27:16.794343 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:27:16.794343 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 27 09:27:16.794343 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:27:16.794343 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 09:27:16.794343 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:27:16.794343 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:27:16.794343 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:27:16.794343 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:27:16.794343 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:27:16.794343 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:27:16.795528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:27:16.795528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:27:16.795528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:27:16.795528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:27:16.795528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:27:16.795528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:27:16.795528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 09:27:16.795528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 09:27:16.795528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 09:27:16.795528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 09:27:16.795528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:27:16.795528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:27:16.795528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:27:16.795528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:27:16.795528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:27:16.795528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:27:16.796435 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:27:16.796435 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:27:16.796435 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:27:16.796435 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:27:16.796435 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:27:16.796435 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:27:16.796435 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:27:16.796435 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:27:16.796435 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:27:16.796435 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:27:16.796435 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 09:27:16.796435 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:27:16.796435 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 27 09:27:16.796435 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:27:16.796435 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 491/967] 10.222.0.70 () {56 vars in 1117 bytes} [Thu Oct 27 09:27:16 2022] GET /identity/v3/services/placement => generated 90 bytes in 143 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:27:16.802127 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641f250d-9aa7-4bbd-802b-2622606ba078 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:27:16.889134 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:27:16.898909 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641f250d-9aa7-4bbd-802b-2622606ba078 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:27:16.936168 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b467341-42de-4a87-9025-f2ccf2e647ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:27:16.936886 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b467341-42de-4a87-9025-f2ccf2e647ad None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:16.937283 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b467341-42de-4a87-9025-f2ccf2e647ad None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:16.937556 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b467341-42de-4a87-9025-f2ccf2e647ad None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:16.938433 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b467341-42de-4a87-9025-f2ccf2e647ad None admin] RBAC: Adding query filter params (name=placement) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:27:16.938736 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b467341-42de-4a87-9025-f2ccf2e647ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:27:16.940606 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b467341-42de-4a87-9025-f2ccf2e647ad None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:27:16.946896 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 477/968] 10.222.0.70 () {56 vars in 1116 bytes} [Thu Oct 27 09:27:16 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:27:16.952912 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c462f45f-51c6-4b17-a875-70e545d463d1 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:27:17.055793 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:27:17.062042 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c462f45f-51c6-4b17-a875-70e545d463d1 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:27:17.093878 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e24a3-a783-41cb-9963-87b5906fbe58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:27:17.095000 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e24a3-a783-41cb-9963-87b5906fbe58 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:17.095274 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e24a3-a783-41cb-9963-87b5906fbe58 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:17.095542 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e24a3-a783-41cb-9963-87b5906fbe58 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:17.096145 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97e24a3-a783-41cb-9963-87b5906fbe58 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:27:17.096441 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97e24a3-a783-41cb-9963-87b5906fbe58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:27:17.097810 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97e24a3-a783-41cb-9963-87b5906fbe58 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:27:17.102141 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 492/969] 10.222.0.70 () {56 vars in 1116 bytes} [Thu Oct 27 09:27:16 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:27:19.252127 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530a0b5-d735-4514-8769-6f927eb13932 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:19.252693 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530a0b5-d735-4514-8769-6f927eb13932 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:19.252693 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530a0b5-d735-4514-8769-6f927eb13932 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:19.254019 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 478/970] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:27:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:27:19.259270 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feba8f4-9e98-410e-921a-bf9c3c2bb4ae None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:19.259661 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feba8f4-9e98-410e-921a-bf9c3c2bb4ae None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:19.259821 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feba8f4-9e98-410e-921a-bf9c3c2bb4ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:19.278137 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-6feba8f4-9e98-410e-921a-bf9c3c2bb4ae None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:27:19.279321 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6feba8f4-9e98-410e-921a-bf9c3c2bb4ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:27:19.344140 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 493/971] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:27:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 09:27:19.613288 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a530a0b5-d735-4514-8769-6f927eb13932 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:27:19.711364 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a530a0b5-d735-4514-8769-6f927eb13932 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:27:19.744211 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c13a82-7038-4f3d-bd74-a9a4273debd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:27:19.744825 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c13a82-7038-4f3d-bd74-a9a4273debd5 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:19.745104 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c13a82-7038-4f3d-bd74-a9a4273debd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:19.745373 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c13a82-7038-4f3d-bd74-a9a4273debd5 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:19.746047 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c13a82-7038-4f3d-bd74-a9a4273debd5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:27:19.747403 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c13a82-7038-4f3d-bd74-a9a4273debd5 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:27:19.764535 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 479/972] 10.222.0.70 () {60 vars in 1141 bytes} [Thu Oct 27 09:27:19 2022] POST /identity/v3/services => generated 246 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:27:21.938905 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edda024-3bfd-484b-8fd5-7169a0014614 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:21.940027 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edda024-3bfd-484b-8fd5-7169a0014614 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:21.940432 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edda024-3bfd-484b-8fd5-7169a0014614 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:21.942055 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 494/973] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:27:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 27 09:27:21.948967 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd4f84-844a-40d0-8f84-26a358c2d0e9 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:21.949517 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd4f84-844a-40d0-8f84-26a358c2d0e9 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:21.949517 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd4f84-844a-40d0-8f84-26a358c2d0e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:21.969381 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-60fd4f84-844a-40d0-8f84-26a358c2d0e9 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:27:21.970601 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-60fd4f84-844a-40d0-8f84-26a358c2d0e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:27:21.979999 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:27:22.041150 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 480/974] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:27:21 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 09:27:22.308424 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edda024-3bfd-484b-8fd5-7169a0014614 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:27:22.395096 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:27:22.405682 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edda024-3bfd-484b-8fd5-7169a0014614 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:27:22.438754 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc73e0-0337-4b44-891e-f04cef229fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:27:22.439367 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc73e0-0337-4b44-891e-f04cef229fc8 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:22.439646 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc73e0-0337-4b44-891e-f04cef229fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:22.439915 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc73e0-0337-4b44-891e-f04cef229fc8 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:22.440488 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcc73e0-0337-4b44-891e-f04cef229fc8 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:27:22.441869 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcc73e0-0337-4b44-891e-f04cef229fc8 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:27:22.450401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-0fcc73e0-0337-4b44-891e-f04cef229fc8 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 27 09:27:22.450401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:27:22.450401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 09:27:22.450401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 09:27:22.450401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 09:27:22.450401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 09:27:22.450401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 09:27:22.450401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:27:22.450401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 27 09:27:22.450401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:27:22.450401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 09:27:22.450401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:27:22.450401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:27:22.450401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:27:22.450401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:27:22.450401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:27:22.450401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:27:22.451647 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:27:22.451647 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:27:22.451647 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:27:22.451647 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:27:22.451647 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:27:22.451647 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:27:22.451647 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 09:27:22.451647 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 09:27:22.451647 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 09:27:22.451647 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 09:27:22.451647 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:27:22.451647 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:27:22.451647 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:27:22.451647 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:27:22.451647 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:27:22.451647 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:27:22.452931 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:27:22.452931 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:27:22.452931 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:27:22.452931 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:27:22.452931 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:27:22.452931 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:27:22.452931 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:27:22.452931 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:27:22.452931 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:27:22.452931 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:27:22.452931 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 09:27:22.452931 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:27:22.452931 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 27 09:27:22.452931 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:27:22.452931 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 495/975] 10.222.0.70 () {56 vars in 1117 bytes} [Thu Oct 27 09:27:22 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:27:22.457403 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fd4f84-844a-40d0-8f84-26a358c2d0e9 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:27:22.551492 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fd4f84-844a-40d0-8f84-26a358c2d0e9 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:27:22.583470 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d0ee7f-2a5e-4861-9bdd-f346ad9b9b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:27:22.584055 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d0ee7f-2a5e-4861-9bdd-f346ad9b9b47 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:22.584324 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d0ee7f-2a5e-4861-9bdd-f346ad9b9b47 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:22.584616 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d0ee7f-2a5e-4861-9bdd-f346ad9b9b47 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:22.585221 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d0ee7f-2a5e-4861-9bdd-f346ad9b9b47 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:27:22.585515 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d0ee7f-2a5e-4861-9bdd-f346ad9b9b47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:27:22.586857 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d0ee7f-2a5e-4861-9bdd-f346ad9b9b47 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:27:22.591952 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 481/976] 10.222.0.70 () {56 vars in 1116 bytes} [Thu Oct 27 09:27:22 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:27:22.598205 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc73e0-0337-4b44-891e-f04cef229fc8 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:27:22.693946 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc73e0-0337-4b44-891e-f04cef229fc8 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:27:22.726902 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2060f597-a69b-4f62-9b5c-a9f807b5303d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:27:22.727490 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060f597-a69b-4f62-9b5c-a9f807b5303d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:22.727766 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060f597-a69b-4f62-9b5c-a9f807b5303d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:22.728038 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060f597-a69b-4f62-9b5c-a9f807b5303d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:22.728741 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2060f597-a69b-4f62-9b5c-a9f807b5303d None admin] RBAC: Adding query filter params (interface=public, service_id=ab8d512f738a4535878f0201daf964a0, region_id=RegionOne) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:27:22.729047 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2060f597-a69b-4f62-9b5c-a9f807b5303d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:27:22.730383 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2060f597-a69b-4f62-9b5c-a9f807b5303d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:27:22.735157 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 496/977] 10.222.0.70 () {56 vars in 1251 bytes} [Thu Oct 27 09:27:22 2022] GET /identity/v3/endpoints?service_id=ab8d512f738a4535878f0201daf964a0&interface=public®ion_id=RegionOne => generated 194 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:27:22.740887 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d0ee7f-2a5e-4861-9bdd-f346ad9b9b47 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:27:22.744549 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:27:22.835972 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d0ee7f-2a5e-4861-9bdd-f346ad9b9b47 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:27:22.868011 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1196dfd5-6b4c-4cca-b4c0-8af294d4a0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:27:22.868618 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1196dfd5-6b4c-4cca-b4c0-8af294d4a0b8 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:22.868895 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1196dfd5-6b4c-4cca-b4c0-8af294d4a0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:22.869159 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1196dfd5-6b4c-4cca-b4c0-8af294d4a0b8 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:22.869739 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1196dfd5-6b4c-4cca-b4c0-8af294d4a0b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:27:22.871080 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1196dfd5-6b4c-4cca-b4c0-8af294d4a0b8 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:27:22.877622 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 482/978] 10.222.0.70 () {56 vars in 1087 bytes} [Thu Oct 27 09:27:22 2022] GET /identity/v3/services => generated 2191 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:27:25.022319 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc3193f-a8b0-49f5-b640-fb14dde651de None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:25.023378 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc3193f-a8b0-49f5-b640-fb14dde651de None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:25.023823 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc3193f-a8b0-49f5-b640-fb14dde651de None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:25.025606 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 497/979] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:27:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:27:25.031964 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134608ca-069d-4cbb-ac1f-dcc23c0804c3 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:25.033164 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134608ca-069d-4cbb-ac1f-dcc23c0804c3 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:25.033164 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134608ca-069d-4cbb-ac1f-dcc23c0804c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:25.055830 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-134608ca-069d-4cbb-ac1f-dcc23c0804c3 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:27:25.057296 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-134608ca-069d-4cbb-ac1f-dcc23c0804c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:27:25.119380 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:27:25.125930 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 483/980] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:27:25 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 27 09:27:25.390052 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc3193f-a8b0-49f5-b640-fb14dde651de None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:27:25.409649 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:27:25.489664 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc3193f-a8b0-49f5-b640-fb14dde651de None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:27:25.521481 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea0d3f4-bb46-4bf1-b751-0289903ee914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:27:25.522093 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea0d3f4-bb46-4bf1-b751-0289903ee914 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:25.522372 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea0d3f4-bb46-4bf1-b751-0289903ee914 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:25.522643 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea0d3f4-bb46-4bf1-b751-0289903ee914 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:25.523208 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea0d3f4-bb46-4bf1-b751-0289903ee914 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:27:25.524565 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea0d3f4-bb46-4bf1-b751-0289903ee914 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:27:25.533661 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-9ea0d3f4-bb46-4bf1-b751-0289903ee914 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 27 09:27:25.533661 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:27:25.533661 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 09:27:25.533661 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 09:27:25.533661 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 09:27:25.533661 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 09:27:25.533661 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 09:27:25.533661 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:27:25.533661 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 27 09:27:25.533661 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:27:25.533661 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 09:27:25.533661 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:27:25.533661 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 09:27:25.533661 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 09:27:25.533661 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 09:27:25.533661 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 09:27:25.533661 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 09:27:25.534948 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 09:27:25.534948 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 09:27:25.534948 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 09:27:25.534948 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 09:27:25.534948 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 09:27:25.534948 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 09:27:25.534948 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 09:27:25.534948 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 09:27:25.534948 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 09:27:25.534948 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 09:27:25.534948 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 09:27:25.534948 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 09:27:25.534948 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 09:27:25.534948 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 09:27:25.534948 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 09:27:25.534948 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 09:27:25.536089 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 09:27:25.536089 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 27 09:27:25.536089 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 09:27:25.536089 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 27 09:27:25.536089 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 09:27:25.536089 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 09:27:25.536089 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 09:27:25.536089 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 27 09:27:25.536089 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 09:27:25.536089 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 27 09:27:25.536089 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 09:27:25.536089 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 09:27:25.536089 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 27 09:27:25.536089 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 27 09:27:25.536089 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 498/981] 10.222.0.70 () {56 vars in 1117 bytes} [Thu Oct 27 09:27:25 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 09:27:25.540605 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134608ca-069d-4cbb-ac1f-dcc23c0804c3 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:27:25.637172 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134608ca-069d-4cbb-ac1f-dcc23c0804c3 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:27:25.669301 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e384259-b4ff-495b-baaa-84972dbb199a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:27:25.669890 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e384259-b4ff-495b-baaa-84972dbb199a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:25.670166 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e384259-b4ff-495b-baaa-84972dbb199a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:25.670437 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e384259-b4ff-495b-baaa-84972dbb199a None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:25.671047 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e384259-b4ff-495b-baaa-84972dbb199a None admin] RBAC: Adding query filter params (name=placement) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:27:25.671345 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e384259-b4ff-495b-baaa-84972dbb199a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:27:25.673267 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e384259-b4ff-495b-baaa-84972dbb199a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:27:25.678172 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 484/982] 10.222.0.70 () {56 vars in 1116 bytes} [Thu Oct 27 09:27:25 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:27:25.683815 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea0d3f4-bb46-4bf1-b751-0289903ee914 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:27:25.763304 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:27:25.780935 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea0d3f4-bb46-4bf1-b751-0289903ee914 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:27:25.816032 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7987c1c3-4ec2-4b2a-8494-dc6d7e1e8cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:27:25.816582 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987c1c3-4ec2-4b2a-8494-dc6d7e1e8cac None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:27:25.816864 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987c1c3-4ec2-4b2a-8494-dc6d7e1e8cac None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:27:25.817141 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987c1c3-4ec2-4b2a-8494-dc6d7e1e8cac None admin] PATH_INFO: `/v3/endpoints` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:27:25.817791 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7987c1c3-4ec2-4b2a-8494-dc6d7e1e8cac None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:27:25.819133 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7987c1c3-4ec2-4b2a-8494-dc6d7e1e8cac None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:27:25.837771 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 499/983] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:27:25 2022] POST /identity/v3/endpoints => generated 331 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:27:35.678430 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=69 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:27:35.833506 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:28:29.437521 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9711ff-34e4-41fd-a03f-abbbb1ee360e None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:28:29.439301 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9711ff-34e4-41fd-a03f-abbbb1ee360e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:28:29.440431 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9711ff-34e4-41fd-a03f-abbbb1ee360e None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:28:29.447618 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 485/984] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:28:29 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:28:29.452797 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4b036-7773-40e9-8abc-097efd0379f1 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:28:29.453214 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4b036-7773-40e9-8abc-097efd0379f1 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:28:29.453434 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4b036-7773-40e9-8abc-097efd0379f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:28:29.551935 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-7cb4b036-7773-40e9-8abc-097efd0379f1 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:28:29.553633 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7cb4b036-7773-40e9-8abc-097efd0379f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:28:29.704458 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 500/985] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:28:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:28:30.003248 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ed057-59ae-4d06-87f7-24e7bb3e5cff None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:28:30.003531 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ed057-59ae-4d06-87f7-24e7bb3e5cff None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:28:30.004052 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ed057-59ae-4d06-87f7-24e7bb3e5cff None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:28:30.005518 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 486/986] 10.222.0.70 () {54 vars in 965 bytes} [Thu Oct 27 09:28:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:28:30.011571 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c02586-57c3-4659-b502-a5d565040bd1 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:28:30.011998 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c02586-57c3-4659-b502-a5d565040bd1 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:28:30.013029 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c02586-57c3-4659-b502-a5d565040bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:28:30.095714 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-d5c02586-57c3-4659-b502-a5d565040bd1 None None] MFA Rules not processed for user `45d8920d9a7940abb112dbbb3d7f6e65`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:28:30.096951 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d5c02586-57c3-4659-b502-a5d565040bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:28:30.120624 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:28:30.191100 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ed057-59ae-4d06-87f7-24e7bb3e5cff None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:28:30.192903 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-538ed057-59ae-4d06-87f7-24e7bb3e5cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:28:30.194692 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 501/987] 10.222.0.70 () {58 vars in 1062 bytes} [Thu Oct 27 09:28:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:28:30.324360 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ed057-59ae-4d06-87f7-24e7bb3e5cff None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:28:30.408861 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f8bea-a95f-42a2-a19a-e847603bc033 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:28:30.409468 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f8bea-a95f-42a2-a19a-e847603bc033 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:28:30.409743 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f8bea-a95f-42a2-a19a-e847603bc033 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:28:30.410013 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f8bea-a95f-42a2-a19a-e847603bc033 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:28:30.411581 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a37f8bea-a95f-42a2-a19a-e847603bc033 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:28:30.470605 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:28:30.529813 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37f8bea-a95f-42a2-a19a-e847603bc033 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:28:30.531254 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37f8bea-a95f-42a2-a19a-e847603bc033 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:28:30.551200 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 487/988] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:28:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 09:28:33.031363 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd3913e-d32d-4769-a2fe-21a07bc58518 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:28:33.031878 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd3913e-d32d-4769-a2fe-21a07bc58518 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:28:33.031878 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd3913e-d32d-4769-a2fe-21a07bc58518 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:28:33.033345 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 502/989] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:28:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:28:33.038347 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5790112e-56d1-4776-b43b-12547653af97 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:28:33.038761 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5790112e-56d1-4776-b43b-12547653af97 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:28:33.038886 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5790112e-56d1-4776-b43b-12547653af97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:28:33.114766 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-5790112e-56d1-4776-b43b-12547653af97 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:28:33.115971 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5790112e-56d1-4776-b43b-12547653af97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:28:33.219572 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 488/990] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:28:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:28:33.506311 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd3913e-d32d-4769-a2fe-21a07bc58518 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:28:33.507895 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ebd3913e-d32d-4769-a2fe-21a07bc58518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:28:33.613445 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd3913e-d32d-4769-a2fe-21a07bc58518 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:28:33.686830 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:28:33.689522 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba83ce9c-9332-4572-ad17-228bb0b52b56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:28:33.690170 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83ce9c-9332-4572-ad17-228bb0b52b56 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:28:33.690453 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83ce9c-9332-4572-ad17-228bb0b52b56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:28:33.690779 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83ce9c-9332-4572-ad17-228bb0b52b56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:28:33.693016 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ba83ce9c-9332-4572-ad17-228bb0b52b56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:28:33.799782 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba83ce9c-9332-4572-ad17-228bb0b52b56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:28:33.801226 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba83ce9c-9332-4572-ad17-228bb0b52b56 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:28:33.818556 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 503/991] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:28:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 09:28:36.260686 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b935a-6d2e-4f11-9af1-0f16e1c53b88 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:28:36.261224 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b935a-6d2e-4f11-9af1-0f16e1c53b88 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:28:36.261224 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b935a-6d2e-4f11-9af1-0f16e1c53b88 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:28:36.263096 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 489/992] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:28:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:28:36.268238 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9810f1fd-6fe1-4d1e-b3da-7a9eaf41d9cc None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:28:36.268669 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9810f1fd-6fe1-4d1e-b3da-7a9eaf41d9cc None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:28:36.268809 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9810f1fd-6fe1-4d1e-b3da-7a9eaf41d9cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:28:36.334933 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-9810f1fd-6fe1-4d1e-b3da-7a9eaf41d9cc None None] MFA Rules not processed for user `483ee2e1417c419482b7963c8e6dd62b`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:28:36.336131 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9810f1fd-6fe1-4d1e-b3da-7a9eaf41d9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:28:36.392283 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:28:36.437233 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 504/993] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:28:36 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:28:36.732317 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a2797-12b0-432b-9ed2-9d170c93f9b0 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:28:36.732630 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a2797-12b0-432b-9ed2-9d170c93f9b0 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:28:36.732902 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a2797-12b0-432b-9ed2-9d170c93f9b0 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:28:36.734261 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 490/994] 10.222.0.70 () {54 vars in 965 bytes} [Thu Oct 27 09:28:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:28:36.739754 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212a8da-540f-480b-9263-5c4a356db886 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:28:36.740035 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212a8da-540f-480b-9263-5c4a356db886 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:28:36.740304 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212a8da-540f-480b-9263-5c4a356db886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:28:36.817005 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-7212a8da-540f-480b-9263-5c4a356db886 None None] MFA Rules not processed for user `45d8920d9a7940abb112dbbb3d7f6e65`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:28:36.818190 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7212a8da-540f-480b-9263-5c4a356db886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:28:36.890949 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 505/995] 10.222.0.70 () {58 vars in 1062 bytes} [Thu Oct 27 09:28:36 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:28:36.899934 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a2797-12b0-432b-9ed2-9d170c93f9b0 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:28:36.901492 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fb1a2797-12b0-432b-9ed2-9d170c93f9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:28:36.912036 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:28:37.006342 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a2797-12b0-432b-9ed2-9d170c93f9b0 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:28:37.078194 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33120fd-8d9a-4fe0-990c-6acfcd2507d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:28:37.078796 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33120fd-8d9a-4fe0-990c-6acfcd2507d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:28:37.079071 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33120fd-8d9a-4fe0-990c-6acfcd2507d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:28:37.079337 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33120fd-8d9a-4fe0-990c-6acfcd2507d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:28:37.080936 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b33120fd-8d9a-4fe0-990c-6acfcd2507d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:28:37.142112 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:28:37.188695 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33120fd-8d9a-4fe0-990c-6acfcd2507d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:28:37.190145 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33120fd-8d9a-4fe0-990c-6acfcd2507d1 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:28:37.208086 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 491/996] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:28:36 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:28:40.243218 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6ca7c0-357d-48ed-afd6-9297f51bfd14 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:28:40.243755 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6ca7c0-357d-48ed-afd6-9297f51bfd14 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:28:40.243755 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6ca7c0-357d-48ed-afd6-9297f51bfd14 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:28:40.245134 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 506/997] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:28:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:28:40.250362 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aaf2e8-83fb-4d6a-a3d3-26853079c4be None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:28:40.250755 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aaf2e8-83fb-4d6a-a3d3-26853079c4be None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:28:40.250892 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aaf2e8-83fb-4d6a-a3d3-26853079c4be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:28:40.316086 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-22aaf2e8-83fb-4d6a-a3d3-26853079c4be None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:28:40.317327 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-22aaf2e8-83fb-4d6a-a3d3-26853079c4be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:28:40.412825 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 492/998] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:28:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:28:40.706055 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6ca7c0-357d-48ed-afd6-9297f51bfd14 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:28:40.707607 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4d6ca7c0-357d-48ed-afd6-9297f51bfd14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:28:40.757598 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:28:40.810399 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6ca7c0-357d-48ed-afd6-9297f51bfd14 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:28:40.881927 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233c4571-1f4a-43fe-8cac-44b93523521d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:28:40.882527 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c4571-1f4a-43fe-8cac-44b93523521d service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:28:40.882800 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c4571-1f4a-43fe-8cac-44b93523521d service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:28:40.883067 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c4571-1f4a-43fe-8cac-44b93523521d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:28:40.884661 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-233c4571-1f4a-43fe-8cac-44b93523521d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:28:40.980328 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:28:40.994238 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233c4571-1f4a-43fe-8cac-44b93523521d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:28:40.995649 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233c4571-1f4a-43fe-8cac-44b93523521d service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:28:41.014021 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 507/999] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:28:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:28:43.830903 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0489e-3ada-4f3e-b1be-f3912d7305d2 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:28:43.831475 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0489e-3ada-4f3e-b1be-f3912d7305d2 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:28:43.831475 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0489e-3ada-4f3e-b1be-f3912d7305d2 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:28:43.833018 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 493/1000] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:28:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:28:43.838630 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f01b2-3ee4-4971-9123-43bd8fd9ace9 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:28:43.839226 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f01b2-3ee4-4971-9123-43bd8fd9ace9 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:28:43.839226 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f01b2-3ee4-4971-9123-43bd8fd9ace9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:28:43.912245 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-a84f01b2-3ee4-4971-9123-43bd8fd9ace9 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:28:43.913518 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a84f01b2-3ee4-4971-9123-43bd8fd9ace9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:28:44.018860 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 508/1001] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:28:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:28:44.305991 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce0489e-3ada-4f3e-b1be-f3912d7305d2 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:28:44.307499 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-bce0489e-3ada-4f3e-b1be-f3912d7305d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:28:44.327719 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:28:44.413235 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce0489e-3ada-4f3e-b1be-f3912d7305d2 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:28:44.487036 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd83a6c7-2a79-4810-8e5a-f44a1df7a7d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:28:44.487634 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83a6c7-2a79-4810-8e5a-f44a1df7a7d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:28:44.487912 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83a6c7-2a79-4810-8e5a-f44a1df7a7d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:28:44.488178 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83a6c7-2a79-4810-8e5a-f44a1df7a7d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:28:44.489787 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fd83a6c7-2a79-4810-8e5a-f44a1df7a7d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:28:44.561062 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:28:44.599991 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd83a6c7-2a79-4810-8e5a-f44a1df7a7d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:28:44.601465 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd83a6c7-2a79-4810-8e5a-f44a1df7a7d2 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:28:44.619404 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 494/1002] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:28:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:28:46.836726 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f007c2-f304-4df9-b31d-23ced334863b None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:28:46.837241 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f007c2-f304-4df9-b31d-23ced334863b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:28:46.837351 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f007c2-f304-4df9-b31d-23ced334863b None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:28:46.838601 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 509/1003] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:28:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:28:46.844154 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655aeb2-aaaf-420b-b15a-551cfd68aa1e None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:28:46.844588 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655aeb2-aaaf-420b-b15a-551cfd68aa1e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:28:46.844814 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655aeb2-aaaf-420b-b15a-551cfd68aa1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:28:46.917529 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-3655aeb2-aaaf-420b-b15a-551cfd68aa1e None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:28:46.919505 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3655aeb2-aaaf-420b-b15a-551cfd68aa1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:28:47.017702 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 495/1004] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:28:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:28:47.298371 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f007c2-f304-4df9-b31d-23ced334863b None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:28:47.299939 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-20f007c2-f304-4df9-b31d-23ced334863b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:28:47.361329 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:28:47.406042 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f007c2-f304-4df9-b31d-23ced334863b None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:28:47.479840 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c40c8f8-8611-4c9a-b75c-8c530538b436 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:28:47.480399 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40c8f8-8611-4c9a-b75c-8c530538b436 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:28:47.480705 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40c8f8-8611-4c9a-b75c-8c530538b436 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:28:47.480977 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40c8f8-8611-4c9a-b75c-8c530538b436 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:28:47.482549 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9c40c8f8-8611-4c9a-b75c-8c530538b436 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:28:47.589378 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:28:47.592387 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c40c8f8-8611-4c9a-b75c-8c530538b436 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:28:47.594365 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c40c8f8-8611-4c9a-b75c-8c530538b436 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:28:47.612441 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 510/1005] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:28:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:28:49.834535 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04b17a0-1449-412a-a03b-c82b7f80030b None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:28:49.835060 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04b17a0-1449-412a-a03b-c82b7f80030b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:28:49.835060 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04b17a0-1449-412a-a03b-c82b7f80030b None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:28:49.836639 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 496/1006] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:28:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:28:49.842734 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8358dc66-1b31-46bd-ad5d-8b63b06a108f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:28:49.843177 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8358dc66-1b31-46bd-ad5d-8b63b06a108f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:28:49.843269 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8358dc66-1b31-46bd-ad5d-8b63b06a108f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:28:49.907692 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-8358dc66-1b31-46bd-ad5d-8b63b06a108f None None] MFA Rules not processed for user `483ee2e1417c419482b7963c8e6dd62b`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:28:49.908909 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8358dc66-1b31-46bd-ad5d-8b63b06a108f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:28:50.004831 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 511/1007] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:28:49 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:28:50.284687 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04b17a0-1449-412a-a03b-c82b7f80030b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:28:50.286026 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e04b17a0-1449-412a-a03b-c82b7f80030b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:28:50.319232 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:28:50.395100 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04b17a0-1449-412a-a03b-c82b7f80030b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:28:50.471762 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fb7909-e047-42d5-b624-d574ed30bebf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:28:50.472368 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb7909-e047-42d5-b624-d574ed30bebf service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:28:50.472682 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb7909-e047-42d5-b624-d574ed30bebf service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:28:50.472952 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb7909-e047-42d5-b624-d574ed30bebf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:28:50.474535 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-12fb7909-e047-42d5-b624-d574ed30bebf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:28:50.552357 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:28:50.584291 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fb7909-e047-42d5-b624-d574ed30bebf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:28:50.585797 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fb7909-e047-42d5-b624-d574ed30bebf service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:28:50.603544 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 497/1008] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:28:50 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:28:53.007258 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3af83b0-3340-451a-99de-02af04bd897d None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:28:53.007846 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3af83b0-3340-451a-99de-02af04bd897d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:28:53.007846 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3af83b0-3340-451a-99de-02af04bd897d None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:28:53.009163 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 512/1009] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:28:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:28:53.014995 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69350822-7b4a-4061-aa2b-a2aa7860fcea None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:28:53.015394 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69350822-7b4a-4061-aa2b-a2aa7860fcea None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:28:53.015534 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69350822-7b4a-4061-aa2b-a2aa7860fcea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:28:53.080972 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-69350822-7b4a-4061-aa2b-a2aa7860fcea None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:28:53.082175 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-69350822-7b4a-4061-aa2b-a2aa7860fcea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:28:53.177468 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 498/1010] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:28:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:28:53.459309 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3af83b0-3340-451a-99de-02af04bd897d None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:28:53.460880 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b3af83b0-3340-451a-99de-02af04bd897d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:28:53.529007 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:28:53.562250 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3af83b0-3340-451a-99de-02af04bd897d None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:28:53.633204 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d3e294-f47b-486f-a7d3-34a0e1265450 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:28:53.633804 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3e294-f47b-486f-a7d3-34a0e1265450 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:28:53.634082 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3e294-f47b-486f-a7d3-34a0e1265450 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:28:53.634350 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3e294-f47b-486f-a7d3-34a0e1265450 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:28:53.635906 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-28d3e294-f47b-486f-a7d3-34a0e1265450 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:28:53.742448 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d3e294-f47b-486f-a7d3-34a0e1265450 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:28:53.743865 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d3e294-f47b-486f-a7d3-34a0e1265450 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:28:53.754185 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:28:53.763227 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 513/1011] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:28:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:28:55.949181 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40cf89e-8c17-4bc2-8b23-cbaec7079dd9 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:28:55.949747 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40cf89e-8c17-4bc2-8b23-cbaec7079dd9 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:28:55.949747 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40cf89e-8c17-4bc2-8b23-cbaec7079dd9 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:28:55.951037 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 499/1012] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:28:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:28:55.956330 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87f718-27ef-417c-98ca-3eb0e4fe5b35 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:28:55.956749 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87f718-27ef-417c-98ca-3eb0e4fe5b35 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:28:55.956917 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87f718-27ef-417c-98ca-3eb0e4fe5b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:28:56.021691 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-ea87f718-27ef-417c-98ca-3eb0e4fe5b35 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:28:56.022874 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ea87f718-27ef-417c-98ca-3eb0e4fe5b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:28:56.114477 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 514/1013] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:28:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:28:56.400572 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40cf89e-8c17-4bc2-8b23-cbaec7079dd9 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:28:56.402089 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f40cf89e-8c17-4bc2-8b23-cbaec7079dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:28:56.442249 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:28:56.506217 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40cf89e-8c17-4bc2-8b23-cbaec7079dd9 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:28:56.578671 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c956085-1ce4-41af-9996-32ac1e778e2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:28:56.579265 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c956085-1ce4-41af-9996-32ac1e778e2b service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:28:56.579536 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c956085-1ce4-41af-9996-32ac1e778e2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:28:56.579797 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c956085-1ce4-41af-9996-32ac1e778e2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:28:56.581377 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0c956085-1ce4-41af-9996-32ac1e778e2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:28:56.661699 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:28:56.686912 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c956085-1ce4-41af-9996-32ac1e778e2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:28:56.688330 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c956085-1ce4-41af-9996-32ac1e778e2b service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:28:56.706135 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 500/1014] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:28:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:28:59.295874 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dfe748-c138-44dc-9424-aef0112f952e None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:28:59.296298 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dfe748-c138-44dc-9424-aef0112f952e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:28:59.296422 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dfe748-c138-44dc-9424-aef0112f952e None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:28:59.297804 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 515/1015] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:28:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:28:59.302970 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047aac8e-8731-4362-9470-91209f95573f None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:28:59.303400 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047aac8e-8731-4362-9470-91209f95573f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:28:59.303500 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047aac8e-8731-4362-9470-91209f95573f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:28:59.369094 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-047aac8e-8731-4362-9470-91209f95573f None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:28:59.370318 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-047aac8e-8731-4362-9470-91209f95573f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:28:59.465198 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 501/1016] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:28:59 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:28:59.747936 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dfe748-c138-44dc-9424-aef0112f952e None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:28:59.749493 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-71dfe748-c138-44dc-9424-aef0112f952e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:28:59.824983 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:28:59.852212 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dfe748-c138-44dc-9424-aef0112f952e None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:28:59.928008 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e5b38e-44e0-4b6c-8764-6507d55a2666 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:28:59.928648 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e5b38e-44e0-4b6c-8764-6507d55a2666 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:28:59.928927 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e5b38e-44e0-4b6c-8764-6507d55a2666 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:28:59.929190 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e5b38e-44e0-4b6c-8764-6507d55a2666 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:28:59.930755 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-10e5b38e-44e0-4b6c-8764-6507d55a2666 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:00.043715 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e5b38e-44e0-4b6c-8764-6507d55a2666 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:29:00.045094 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e5b38e-44e0-4b6c-8764-6507d55a2666 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:29:00.063581 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 516/1017] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:28:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:29:04.194339 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491abce1-992e-46b1-ad2d-9f3418973e51 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:04.194860 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491abce1-992e-46b1-ad2d-9f3418973e51 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:04.194860 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491abce1-992e-46b1-ad2d-9f3418973e51 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:04.196296 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 502/1018] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:29:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:29:04.201989 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22972c5-e51e-4b7f-9086-1e724841acf9 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:04.202480 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22972c5-e51e-4b7f-9086-1e724841acf9 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:04.202599 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22972c5-e51e-4b7f-9086-1e724841acf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:04.207912 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:29:04.272636 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-c22972c5-e51e-4b7f-9086-1e724841acf9 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:29:04.273803 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c22972c5-e51e-4b7f-9086-1e724841acf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:04.370623 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 517/1019] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:29:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:29:04.656121 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491abce1-992e-46b1-ad2d-9f3418973e51 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:29:04.657769 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-491abce1-992e-46b1-ad2d-9f3418973e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:04.707761 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:29:04.763245 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491abce1-992e-46b1-ad2d-9f3418973e51 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:29:04.836341 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584fd106-5402-4623-ad1e-3ffc88012510 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:29:04.836970 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584fd106-5402-4623-ad1e-3ffc88012510 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:04.837248 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584fd106-5402-4623-ad1e-3ffc88012510 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:04.837522 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584fd106-5402-4623-ad1e-3ffc88012510 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:04.839195 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-584fd106-5402-4623-ad1e-3ffc88012510 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:04.935547 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:29:04.952718 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584fd106-5402-4623-ad1e-3ffc88012510 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:29:04.954167 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584fd106-5402-4623-ad1e-3ffc88012510 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:29:04.973071 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 503/1020] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:29:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:29:07.764441 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d15725-c3ff-4662-8e2a-62ea0a0771b4 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:07.765000 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d15725-c3ff-4662-8e2a-62ea0a0771b4 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:07.765000 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d15725-c3ff-4662-8e2a-62ea0a0771b4 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:07.766493 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 518/1021] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:29:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:29:07.771463 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027fb792-f8d1-4ed1-ae17-f5323634daa6 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:07.771861 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027fb792-f8d1-4ed1-ae17-f5323634daa6 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:07.772023 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027fb792-f8d1-4ed1-ae17-f5323634daa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:07.840256 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-027fb792-f8d1-4ed1-ae17-f5323634daa6 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:29:07.841502 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-027fb792-f8d1-4ed1-ae17-f5323634daa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:07.941997 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 504/1022] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:29:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:29:08.232485 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d15725-c3ff-4662-8e2a-62ea0a0771b4 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:29:08.234045 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-72d15725-c3ff-4662-8e2a-62ea0a0771b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:08.328048 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:29:08.345113 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d15725-c3ff-4662-8e2a-62ea0a0771b4 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:29:08.420217 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252eeaaa-4a94-4f67-9279-57db7c7ec087 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:29:08.420767 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252eeaaa-4a94-4f67-9279-57db7c7ec087 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:08.421039 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252eeaaa-4a94-4f67-9279-57db7c7ec087 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:08.421302 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252eeaaa-4a94-4f67-9279-57db7c7ec087 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:08.422882 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-252eeaaa-4a94-4f67-9279-57db7c7ec087 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:08.535560 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252eeaaa-4a94-4f67-9279-57db7c7ec087 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:29:08.537020 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252eeaaa-4a94-4f67-9279-57db7c7ec087 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:29:08.555075 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 519/1023] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:29:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:29:12.862575 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae622544-f5f2-4197-8b9d-27f2abdb8bbc None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:12.863024 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae622544-f5f2-4197-8b9d-27f2abdb8bbc None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:12.863133 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae622544-f5f2-4197-8b9d-27f2abdb8bbc None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:12.864446 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 505/1024] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:29:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:29:12.870390 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68e459-548a-4551-957a-4e701f39c096 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:12.870838 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68e459-548a-4551-957a-4e701f39c096 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:12.871057 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68e459-548a-4551-957a-4e701f39c096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:12.886241 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:29:12.938464 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-ea68e459-548a-4551-957a-4e701f39c096 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:29:12.939966 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ea68e459-548a-4551-957a-4e701f39c096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:13.040088 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 520/1025] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:29:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:29:13.321364 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae622544-f5f2-4197-8b9d-27f2abdb8bbc None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:29:13.322917 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ae622544-f5f2-4197-8b9d-27f2abdb8bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:13.382310 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:29:13.430452 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae622544-f5f2-4197-8b9d-27f2abdb8bbc None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:29:13.505821 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e2856f-e57c-4741-a6b0-28a5a3d55adf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:29:13.506425 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2856f-e57c-4741-a6b0-28a5a3d55adf service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:13.506702 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2856f-e57c-4741-a6b0-28a5a3d55adf service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:13.506971 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2856f-e57c-4741-a6b0-28a5a3d55adf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:13.508593 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d9e2856f-e57c-4741-a6b0-28a5a3d55adf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:13.613843 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:29:13.617882 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e2856f-e57c-4741-a6b0-28a5a3d55adf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:29:13.619318 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e2856f-e57c-4741-a6b0-28a5a3d55adf service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:29:13.637539 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 506/1026] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:29:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:29:18.594878 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceff1e4-d2c8-4818-a43b-2e4b8e39f7a9 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:18.595314 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceff1e4-d2c8-4818-a43b-2e4b8e39f7a9 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:18.595410 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceff1e4-d2c8-4818-a43b-2e4b8e39f7a9 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:18.597080 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 521/1027] 10.222.0.70 () {54 vars in 921 bytes} [Thu Oct 27 09:29:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:29:18.602032 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8fabb8-6e6d-4ec4-81c7-ba45bdb9af1d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:18.602444 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8fabb8-6e6d-4ec4-81c7-ba45bdb9af1d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:18.602639 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8fabb8-6e6d-4ec4-81c7-ba45bdb9af1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:18.714751 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-0c8fabb8-6e6d-4ec4-81c7-ba45bdb9af1d None None] MFA Rules not processed for user `2316b30a6e7b4175bd7b90583584eab5`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:29:18.716337 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0c8fabb8-6e6d-4ec4-81c7-ba45bdb9af1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:18.875988 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 507/1028] 10.222.0.70 () {58 vars in 1018 bytes} [Thu Oct 27 09:29:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:29:22.997419 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80dcd08-2c4f-4056-8410-bf0cb5d7d992 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:22.998005 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80dcd08-2c4f-4056-8410-bf0cb5d7d992 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:22.998005 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80dcd08-2c4f-4056-8410-bf0cb5d7d992 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:22.999178 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 522/1029] 10.222.0.70 () {54 vars in 921 bytes} [Thu Oct 27 09:29:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:29:23.004249 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941a74a2-e6a7-4d4a-9c4b-19536cd2c4a1 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:23.004680 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941a74a2-e6a7-4d4a-9c4b-19536cd2c4a1 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:23.004907 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941a74a2-e6a7-4d4a-9c4b-19536cd2c4a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:23.038166 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=59 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:29:23.064800 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:29:23.091894 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-941a74a2-e6a7-4d4a-9c4b-19536cd2c4a1 None None] MFA Rules not processed for user `2316b30a6e7b4175bd7b90583584eab5`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:29:23.093196 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-941a74a2-e6a7-4d4a-9c4b-19536cd2c4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:23.207088 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 508/1030] 10.222.0.70 () {58 vars in 1018 bytes} [Thu Oct 27 09:29:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:29:24.550443 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5761c-eedd-4783-afeb-f3633d06a5b1 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:24.550944 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5761c-eedd-4783-afeb-f3633d06a5b1 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:24.550944 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5761c-eedd-4783-afeb-f3633d06a5b1 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:24.552233 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 523/1031] 10.222.0.70 () {54 vars in 921 bytes} [Thu Oct 27 09:29:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:29:24.557837 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba13f45-0943-4380-8ac1-9fb33165e814 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:24.558251 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba13f45-0943-4380-8ac1-9fb33165e814 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:24.558513 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba13f45-0943-4380-8ac1-9fb33165e814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:24.636610 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-9ba13f45-0943-4380-8ac1-9fb33165e814 None None] MFA Rules not processed for user `2316b30a6e7b4175bd7b90583584eab5`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:29:24.637927 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9ba13f45-0943-4380-8ac1-9fb33165e814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:24.679048 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:29:24.744340 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 509/1032] 10.222.0.70 () {58 vars in 1018 bytes} [Thu Oct 27 09:29:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:29:28.390581 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968f2ee-34e7-402b-b72d-5e1987547aea None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:28.391087 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968f2ee-34e7-402b-b72d-5e1987547aea None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:28.391087 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968f2ee-34e7-402b-b72d-5e1987547aea None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:28.394712 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 524/1033] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:29:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:29:28.399608 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80014cae-d3bf-4d10-a16f-cd5e0523a4fd None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:28.400646 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80014cae-d3bf-4d10-a16f-cd5e0523a4fd None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:28.400941 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80014cae-d3bf-4d10-a16f-cd5e0523a4fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:28.491725 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-80014cae-d3bf-4d10-a16f-cd5e0523a4fd None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:29:28.492865 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-80014cae-d3bf-4d10-a16f-cd5e0523a4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:28.597289 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 510/1034] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:29:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:29:28.905393 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371684e7-581d-4bea-ba47-951b79ab39b1 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:28.906069 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371684e7-581d-4bea-ba47-951b79ab39b1 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:28.906710 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371684e7-581d-4bea-ba47-951b79ab39b1 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:28.908108 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 525/1035] 10.222.0.70 () {54 vars in 961 bytes} [Thu Oct 27 09:29:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:29:28.927186 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab8968-ddbe-431b-b021-3f41e3c2b201 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:28.927484 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab8968-ddbe-431b-b021-3f41e3c2b201 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:28.927818 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab8968-ddbe-431b-b021-3f41e3c2b201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:28.949685 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:29:29.016770 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-eeab8968-ddbe-431b-b021-3f41e3c2b201 None None] MFA Rules not processed for user `dbe4a6e4b6184826b30ac4fc13ad9b1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:29:29.017959 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-eeab8968-ddbe-431b-b021-3f41e3c2b201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:29.145003 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 511/1036] 10.222.0.70 () {58 vars in 1058 bytes} [Thu Oct 27 09:29:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:29:29.153287 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371684e7-581d-4bea-ba47-951b79ab39b1 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:29:29.154934 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-371684e7-581d-4bea-ba47-951b79ab39b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:29.318422 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371684e7-581d-4bea-ba47-951b79ab39b1 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:29:29.396738 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:29:29.440261 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522bac58-a701-4b83-966f-30184b58fe87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe4a6e4b6184826b30ac4fc13ad9b1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:29:29.441103 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522bac58-a701-4b83-966f-30184b58fe87 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:29.441484 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522bac58-a701-4b83-966f-30184b58fe87 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:29.441878 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522bac58-a701-4b83-966f-30184b58fe87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:29.443525 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-522bac58-a701-4b83-966f-30184b58fe87 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:29.542662 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522bac58-a701-4b83-966f-30184b58fe87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:29:29.544412 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522bac58-a701-4b83-966f-30184b58fe87 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:29:29.563927 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 526/1037] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:29:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:29:31.960024 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5fbc06-2f0d-459f-8899-22cdbff2f9b5 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:31.960451 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5fbc06-2f0d-459f-8899-22cdbff2f9b5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:31.960881 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5fbc06-2f0d-459f-8899-22cdbff2f9b5 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:31.962501 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 512/1038] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:29:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:29:31.967865 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb9c40-75f7-4811-8e16-e4623e8620d4 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:31.968555 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb9c40-75f7-4811-8e16-e4623e8620d4 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:31.968555 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb9c40-75f7-4811-8e16-e4623e8620d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:32.035867 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-78eb9c40-75f7-4811-8e16-e4623e8620d4 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:29:32.037080 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-78eb9c40-75f7-4811-8e16-e4623e8620d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:32.077009 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:29:32.141448 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 527/1039] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:29:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:29:32.428816 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5fbc06-2f0d-459f-8899-22cdbff2f9b5 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:29:32.430490 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4c5fbc06-2f0d-459f-8899-22cdbff2f9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:32.499522 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:29:32.605670 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5fbc06-2f0d-459f-8899-22cdbff2f9b5 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:29:32.752053 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba68c8e-f899-448f-a866-c07345a59781 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe4a6e4b6184826b30ac4fc13ad9b1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:29:32.752712 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba68c8e-f899-448f-a866-c07345a59781 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:32.752995 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba68c8e-f899-448f-a866-c07345a59781 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:32.753267 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba68c8e-f899-448f-a866-c07345a59781 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:32.754845 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1ba68c8e-f899-448f-a866-c07345a59781 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:32.779150 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:29:32.851306 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba68c8e-f899-448f-a866-c07345a59781 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:29:32.853622 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba68c8e-f899-448f-a866-c07345a59781 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:29:32.871502 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 513/1040] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:29:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:29:35.375902 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636ee19-b0d5-4fc5-812a-e759553c4863 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:35.376480 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636ee19-b0d5-4fc5-812a-e759553c4863 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:35.376480 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636ee19-b0d5-4fc5-812a-e759553c4863 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:35.377840 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 528/1041] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:29:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:29:35.383187 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb448b8c-3e6a-4e6e-a472-927d723e2127 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:35.383618 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb448b8c-3e6a-4e6e-a472-927d723e2127 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:35.383798 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb448b8c-3e6a-4e6e-a472-927d723e2127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:35.454769 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-eb448b8c-3e6a-4e6e-a472-927d723e2127 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:29:35.455987 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-eb448b8c-3e6a-4e6e-a472-927d723e2127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:35.556931 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:29:35.567011 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 514/1042] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:29:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:29:35.847905 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff2f931-d098-4e2c-b5f0-1bc23ac4dcde None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:35.848310 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff2f931-d098-4e2c-b5f0-1bc23ac4dcde None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:35.848727 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff2f931-d098-4e2c-b5f0-1bc23ac4dcde None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:35.850399 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 529/1043] 10.222.0.70 () {54 vars in 961 bytes} [Thu Oct 27 09:29:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:29:35.856203 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45db0fe9-977c-4ac4-96cb-b2a0adc084cf None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:35.856493 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45db0fe9-977c-4ac4-96cb-b2a0adc084cf None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:35.856817 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45db0fe9-977c-4ac4-96cb-b2a0adc084cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:35.938809 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-45db0fe9-977c-4ac4-96cb-b2a0adc084cf None None] MFA Rules not processed for user `dbe4a6e4b6184826b30ac4fc13ad9b1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:29:35.940397 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-45db0fe9-977c-4ac4-96cb-b2a0adc084cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:36.072924 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 515/1044] 10.222.0.70 () {58 vars in 1058 bytes} [Thu Oct 27 09:29:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:29:36.080694 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff2f931-d098-4e2c-b5f0-1bc23ac4dcde None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:29:36.082292 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4ff2f931-d098-4e2c-b5f0-1bc23ac4dcde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:36.250533 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff2f931-d098-4e2c-b5f0-1bc23ac4dcde None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:29:36.277678 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:29:36.383700 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff022ce7-d4dc-4de6-889f-e973617fea4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe4a6e4b6184826b30ac4fc13ad9b1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:29:36.384924 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff022ce7-d4dc-4de6-889f-e973617fea4a service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:36.385167 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff022ce7-d4dc-4de6-889f-e973617fea4a service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:36.385388 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff022ce7-d4dc-4de6-889f-e973617fea4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:36.386968 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ff022ce7-d4dc-4de6-889f-e973617fea4a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:36.486911 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff022ce7-d4dc-4de6-889f-e973617fea4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:29:36.488495 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff022ce7-d4dc-4de6-889f-e973617fea4a service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:29:36.507754 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 530/1045] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:29:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:29:36.547337 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67bd374-63ee-460f-a660-bd5e20df79a6 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:36.547922 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67bd374-63ee-460f-a660-bd5e20df79a6 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:36.547922 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67bd374-63ee-460f-a660-bd5e20df79a6 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:36.549252 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 516/1046] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:29:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:29:36.554521 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076279e-0d94-4ddf-8de9-70bfe4057448 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:36.554868 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076279e-0d94-4ddf-8de9-70bfe4057448 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:36.555082 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076279e-0d94-4ddf-8de9-70bfe4057448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:36.590506 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:29:36.641866 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-7076279e-0d94-4ddf-8de9-70bfe4057448 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:29:36.643230 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7076279e-0d94-4ddf-8de9-70bfe4057448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:36.766921 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 531/1047] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:29:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:29:37.066192 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac3b447-1de1-40bf-8e3a-0d4a799410da None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:37.066594 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac3b447-1de1-40bf-8e3a-0d4a799410da None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:37.066997 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac3b447-1de1-40bf-8e3a-0d4a799410da None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:37.068675 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 517/1048] 10.222.0.70 () {54 vars in 963 bytes} [Thu Oct 27 09:29:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:29:37.075281 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d3c60-647e-4540-b72b-94489196f9da None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:37.075281 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d3c60-647e-4540-b72b-94489196f9da None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:37.075281 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d3c60-647e-4540-b72b-94489196f9da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:37.155039 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-835d3c60-647e-4540-b72b-94489196f9da None None] MFA Rules not processed for user `5b8e13ce967449c1b12783183e953579`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:29:37.156245 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-835d3c60-647e-4540-b72b-94489196f9da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:37.214204 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:29:37.246734 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 532/1049] 10.222.0.70 () {58 vars in 1060 bytes} [Thu Oct 27 09:29:37 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:29:37.254209 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac3b447-1de1-40bf-8e3a-0d4a799410da None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:29:37.255736 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6ac3b447-1de1-40bf-8e3a-0d4a799410da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:37.317770 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:29:37.376001 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac3b447-1de1-40bf-8e3a-0d4a799410da None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:29:37.456995 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ec973-a3ab-4b3d-a816-f4e2350fe5db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:29:37.457605 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ec973-a3ab-4b3d-a816-f4e2350fe5db service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:37.457885 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ec973-a3ab-4b3d-a816-f4e2350fe5db service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:37.458151 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ec973-a3ab-4b3d-a816-f4e2350fe5db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:37.459742 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-758ec973-a3ab-4b3d-a816-f4e2350fe5db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:37.580015 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:29:37.593256 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758ec973-a3ab-4b3d-a816-f4e2350fe5db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:29:37.594835 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758ec973-a3ab-4b3d-a816-f4e2350fe5db service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:29:37.614383 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 518/1050] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:29:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:29:38.759660 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf1a93-cc41-4f84-a034-24b60e171590 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:38.760539 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf1a93-cc41-4f84-a034-24b60e171590 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:38.760833 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf1a93-cc41-4f84-a034-24b60e171590 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:38.765415 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 533/1051] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:29:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:29:38.767590 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca2041-d87b-4b12-ad65-33f0d039f557 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:38.768023 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca2041-d87b-4b12-ad65-33f0d039f557 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:38.768188 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca2041-d87b-4b12-ad65-33f0d039f557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:38.836470 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-4fca2041-d87b-4b12-ad65-33f0d039f557 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:29:38.837711 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4fca2041-d87b-4b12-ad65-33f0d039f557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:38.936172 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 519/1052] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:29:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:29:39.234211 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bf1a93-cc41-4f84-a034-24b60e171590 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:29:39.235902 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-01bf1a93-cc41-4f84-a034-24b60e171590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:39.390953 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bf1a93-cc41-4f84-a034-24b60e171590 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:29:39.446370 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:29:39.510025 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a56d098-19e1-437b-b334-7dbecb3acdc1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe4a6e4b6184826b30ac4fc13ad9b1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:29:39.510638 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a56d098-19e1-437b-b334-7dbecb3acdc1 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:39.510910 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a56d098-19e1-437b-b334-7dbecb3acdc1 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:39.511176 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a56d098-19e1-437b-b334-7dbecb3acdc1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:39.512828 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7a56d098-19e1-437b-b334-7dbecb3acdc1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:39.614221 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a56d098-19e1-437b-b334-7dbecb3acdc1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:29:39.615829 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a56d098-19e1-437b-b334-7dbecb3acdc1 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:29:39.636215 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 534/1053] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:29:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:29:41.863121 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ec34e-2aa3-4411-86ce-9baa50319aa1 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:41.863694 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ec34e-2aa3-4411-86ce-9baa50319aa1 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:41.863942 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ec34e-2aa3-4411-86ce-9baa50319aa1 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:41.865583 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 520/1054] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:29:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:29:41.870837 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90161eee-9e21-4c48-a59b-fb5d07665233 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:41.871260 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90161eee-9e21-4c48-a59b-fb5d07665233 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:41.871436 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90161eee-9e21-4c48-a59b-fb5d07665233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:41.946652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-90161eee-9e21-4c48-a59b-fb5d07665233 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:29:41.948489 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-90161eee-9e21-4c48-a59b-fb5d07665233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:41.956666 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:29:42.053150 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 535/1055] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:29:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:29:42.342836 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ec34e-2aa3-4411-86ce-9baa50319aa1 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:29:42.344340 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-448ec34e-2aa3-4411-86ce-9baa50319aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:42.408363 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:29:42.499229 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ec34e-2aa3-4411-86ce-9baa50319aa1 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:29:42.618333 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8097af4b-2490-4d4f-954a-71d076105c0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe4a6e4b6184826b30ac4fc13ad9b1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:29:42.618948 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8097af4b-2490-4d4f-954a-71d076105c0d service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:42.619208 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8097af4b-2490-4d4f-954a-71d076105c0d service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:42.619476 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8097af4b-2490-4d4f-954a-71d076105c0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:42.621101 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8097af4b-2490-4d4f-954a-71d076105c0d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:42.641297 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:29:42.721402 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8097af4b-2490-4d4f-954a-71d076105c0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:29:42.723056 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8097af4b-2490-4d4f-954a-71d076105c0d service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:29:42.742255 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 521/1056] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:29:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:29:44.960187 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270e78e-5cac-40d0-af8c-fa24c5a3d57d None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:44.960751 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270e78e-5cac-40d0-af8c-fa24c5a3d57d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:44.960751 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270e78e-5cac-40d0-af8c-fa24c5a3d57d None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:44.963020 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 536/1057] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:29:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:29:44.968899 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ebac7-aa93-4341-9fb3-edd1b6be6364 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:44.969400 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ebac7-aa93-4341-9fb3-edd1b6be6364 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:44.969400 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ebac7-aa93-4341-9fb3-edd1b6be6364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:45.044311 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-dd0ebac7-aa93-4341-9fb3-edd1b6be6364 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:29:45.045596 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-dd0ebac7-aa93-4341-9fb3-edd1b6be6364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:45.153561 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:29:45.161317 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 522/1058] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:29:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:29:45.457881 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f270e78e-5cac-40d0-af8c-fa24c5a3d57d None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:29:45.459400 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f270e78e-5cac-40d0-af8c-fa24c5a3d57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:45.613419 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:29:45.623934 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f270e78e-5cac-40d0-af8c-fa24c5a3d57d None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:29:45.745346 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceb9ee5-7f96-4ccb-8ed8-18db82a8925b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe4a6e4b6184826b30ac4fc13ad9b1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:29:45.745878 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb9ee5-7f96-4ccb-8ed8-18db82a8925b service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:45.746153 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb9ee5-7f96-4ccb-8ed8-18db82a8925b service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:45.746420 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb9ee5-7f96-4ccb-8ed8-18db82a8925b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:45.747991 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6ceb9ee5-7f96-4ccb-8ed8-18db82a8925b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:45.836896 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceb9ee5-7f96-4ccb-8ed8-18db82a8925b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:29:45.838555 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceb9ee5-7f96-4ccb-8ed8-18db82a8925b service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:29:45.849319 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:29:45.859180 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 537/1059] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:29:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:29:48.056387 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99505a6-3a0b-40e1-9b30-3fce1291d1c0 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:48.056921 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99505a6-3a0b-40e1-9b30-3fce1291d1c0 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:48.056921 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99505a6-3a0b-40e1-9b30-3fce1291d1c0 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:48.058228 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 523/1060] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:29:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:29:48.064055 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90752ec2-eaec-4a4b-a5b4-6408dd5da411 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:48.064487 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90752ec2-eaec-4a4b-a5b4-6408dd5da411 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:48.064662 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90752ec2-eaec-4a4b-a5b4-6408dd5da411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:48.132811 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-90752ec2-eaec-4a4b-a5b4-6408dd5da411 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:29:48.134032 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-90752ec2-eaec-4a4b-a5b4-6408dd5da411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:48.234646 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 538/1061] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:29:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:29:48.542725 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99505a6-3a0b-40e1-9b30-3fce1291d1c0 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:29:48.544308 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d99505a6-3a0b-40e1-9b30-3fce1291d1c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:48.707683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99505a6-3a0b-40e1-9b30-3fce1291d1c0 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:29:48.795356 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:29:48.846363 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf90f46-3e61-4ff3-b407-9c7543b1f33c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe4a6e4b6184826b30ac4fc13ad9b1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:29:48.846965 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf90f46-3e61-4ff3-b407-9c7543b1f33c service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:48.847247 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf90f46-3e61-4ff3-b407-9c7543b1f33c service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:48.847514 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf90f46-3e61-4ff3-b407-9c7543b1f33c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:48.849128 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ebf90f46-3e61-4ff3-b407-9c7543b1f33c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:48.957055 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf90f46-3e61-4ff3-b407-9c7543b1f33c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:29:48.959023 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf90f46-3e61-4ff3-b407-9c7543b1f33c service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:29:48.978194 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 524/1062] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:29:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:29:51.445444 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06156ab-393c-44f3-b765-0779c8e53b86 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:51.446551 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06156ab-393c-44f3-b765-0779c8e53b86 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:51.446672 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06156ab-393c-44f3-b765-0779c8e53b86 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:51.448027 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 539/1063] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:29:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:29:51.453543 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ccb6f7-352d-49cf-afef-5c3edbbf5c7c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:51.453994 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ccb6f7-352d-49cf-afef-5c3edbbf5c7c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:51.454230 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ccb6f7-352d-49cf-afef-5c3edbbf5c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:51.521644 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-c0ccb6f7-352d-49cf-afef-5c3edbbf5c7c None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:29:51.522843 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c0ccb6f7-352d-49cf-afef-5c3edbbf5c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:51.555712 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:29:51.622899 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 525/1064] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:29:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:29:51.912723 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06156ab-393c-44f3-b765-0779c8e53b86 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:29:51.914246 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c06156ab-393c-44f3-b765-0779c8e53b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:51.990462 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:29:52.060651 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06156ab-393c-44f3-b765-0779c8e53b86 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:29:52.172423 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dbdfaf-de6c-4495-b733-386575629446 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe4a6e4b6184826b30ac4fc13ad9b1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:29:52.173079 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dbdfaf-de6c-4495-b733-386575629446 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:52.173359 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dbdfaf-de6c-4495-b733-386575629446 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:52.173630 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dbdfaf-de6c-4495-b733-386575629446 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:52.175201 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-08dbdfaf-de6c-4495-b733-386575629446 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:52.213174 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:29:52.264094 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dbdfaf-de6c-4495-b733-386575629446 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:29:52.265716 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dbdfaf-de6c-4495-b733-386575629446 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:29:52.286946 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 540/1065] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:29:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:29:52.873369 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da79fec-6b70-45cd-ba9b-59841ff5cb58 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:52.873915 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da79fec-6b70-45cd-ba9b-59841ff5cb58 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:52.873915 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da79fec-6b70-45cd-ba9b-59841ff5cb58 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:52.875297 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 526/1066] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:29:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:29:52.880672 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cecf55-ef6b-4b3b-99f9-14dc82f6d515 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:52.881148 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cecf55-ef6b-4b3b-99f9-14dc82f6d515 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:52.881249 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cecf55-ef6b-4b3b-99f9-14dc82f6d515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:52.948101 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-e7cecf55-ef6b-4b3b-99f9-14dc82f6d515 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:29:52.949584 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e7cecf55-ef6b-4b3b-99f9-14dc82f6d515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:53.047014 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 541/1067] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:29:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:29:53.351289 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d2673-2b80-416a-a27d-b44cab5f8e1d None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:53.351572 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d2673-2b80-416a-a27d-b44cab5f8e1d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:53.352409 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d2673-2b80-416a-a27d-b44cab5f8e1d None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:53.353765 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 527/1068] 10.222.0.70 () {54 vars in 962 bytes} [Thu Oct 27 09:29:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:29:53.358977 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a54a3-db0a-4847-a5b7-326a2870d440 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:53.359263 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a54a3-db0a-4847-a5b7-326a2870d440 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:53.359531 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a54a3-db0a-4847-a5b7-326a2870d440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:53.375702 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:29:53.435599 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-b03a54a3-db0a-4847-a5b7-326a2870d440 None None] MFA Rules not processed for user `72b0538f0e42446887b1d95b2fb41fb2`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:29:53.436981 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b03a54a3-db0a-4847-a5b7-326a2870d440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:53.511600 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 542/1069] 10.222.0.70 () {58 vars in 1059 bytes} [Thu Oct 27 09:29:53 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:29:53.519581 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85d2673-2b80-416a-a27d-b44cab5f8e1d None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:29:53.521313 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a85d2673-2b80-416a-a27d-b44cab5f8e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:53.625712 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85d2673-2b80-416a-a27d-b44cab5f8e1d None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:29:53.693559 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:29:53.709134 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff36a51-dee4-4711-a1e3-66a7bbec5016 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72b0538f0e42446887b1d95b2fb41fb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:29:53.709768 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff36a51-dee4-4711-a1e3-66a7bbec5016 service glance] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:53.710048 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff36a51-dee4-4711-a1e3-66a7bbec5016 service glance] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:53.710313 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff36a51-dee4-4711-a1e3-66a7bbec5016 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:53.712419 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3ff36a51-dee4-4711-a1e3-66a7bbec5016 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:53.822257 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff36a51-dee4-4711-a1e3-66a7bbec5016 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:29:53.823623 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff36a51-dee4-4711-a1e3-66a7bbec5016 service glance] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:29:53.842172 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 528/1070] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:29:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:29:54.489024 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc1285-059a-4f20-8573-6f2b6ce5f8d5 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:54.489609 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc1285-059a-4f20-8573-6f2b6ce5f8d5 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:54.489609 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc1285-059a-4f20-8573-6f2b6ce5f8d5 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:54.490908 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 543/1071] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:29:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:29:54.497221 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f30e185-ebec-4c32-99ef-3423f009c172 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:54.497705 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f30e185-ebec-4c32-99ef-3423f009c172 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:54.497814 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f30e185-ebec-4c32-99ef-3423f009c172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:54.574646 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-6f30e185-ebec-4c32-99ef-3423f009c172 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:29:54.575834 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6f30e185-ebec-4c32-99ef-3423f009c172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:54.578751 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db717558-0216-44f5-b222-e34bf34242cd None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:54.579101 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db717558-0216-44f5-b222-e34bf34242cd None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:54.579440 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db717558-0216-44f5-b222-e34bf34242cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:54.618316 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:29:54.648778 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-db717558-0216-44f5-b222-e34bf34242cd None None] MFA Rules not processed for user `7f389510a6a044d8a7487561b169544b`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:29:54.650033 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-db717558-0216-44f5-b222-e34bf34242cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:54.674485 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 529/1072] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:29:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:29:54.704398 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:29:54.735870 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 544/1073] 10.222.0.70 () {58 vars in 1009 bytes} [Thu Oct 27 09:29:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:29:54.783665 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7675d5-8ca4-4174-835f-b09fd4b0cd1c None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:54.783998 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7675d5-8ca4-4174-835f-b09fd4b0cd1c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:54.784276 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7675d5-8ca4-4174-835f-b09fd4b0cd1c None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:54.785681 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 530/1074] 10.222.0.70 () {54 vars in 943 bytes} [Thu Oct 27 09:29:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:29:54.792889 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae31b95-e617-4ad0-8eaa-a9f526bca398 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:54.793178 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae31b95-e617-4ad0-8eaa-a9f526bca398 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:54.793453 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae31b95-e617-4ad0-8eaa-a9f526bca398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:54.871259 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-2ae31b95-e617-4ad0-8eaa-a9f526bca398 None None] MFA Rules not processed for user `eecf79ea4700496c89b1a67c614b34d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:29:54.872377 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2ae31b95-e617-4ad0-8eaa-a9f526bca398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:54.968908 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7675d5-8ca4-4174-835f-b09fd4b0cd1c None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:29:54.970428 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7b7675d5-8ca4-4174-835f-b09fd4b0cd1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:54.982276 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 545/1075] 10.222.0.70 () {58 vars in 1040 bytes} [Thu Oct 27 09:29:54 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:29:54.991401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae31b95-e617-4ad0-8eaa-a9f526bca398 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:29:55.000770 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae31b95-e617-4ad0-8eaa-a9f526bca398 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:29:55.027228 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:29:55.106068 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc9f411-b733-411a-9289-ea2a8f5b09dc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eecf79ea4700496c89b1a67c614b34d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:29:55.106675 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9f411-b733-411a-9289-ea2a8f5b09dc service swift] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:55.106954 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9f411-b733-411a-9289-ea2a8f5b09dc service swift] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:55.107223 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9f411-b733-411a-9289-ea2a8f5b09dc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:55.108890 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-efc9f411-b733-411a-9289-ea2a8f5b09dc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:55.121197 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7675d5-8ca4-4174-835f-b09fd4b0cd1c None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:29:55.147073 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:29:55.192051 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc9f411-b733-411a-9289-ea2a8f5b09dc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:29:55.193686 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc9f411-b733-411a-9289-ea2a8f5b09dc service swift] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:29:55.202674 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 546/1076] 10.222.0.70 () {60 vars in 1384 bytes} [Thu Oct 27 09:29:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 27 09:29:55.241528 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d80a2b0-01e8-47ce-ae6c-fde41d7545c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe4a6e4b6184826b30ac4fc13ad9b1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:29:55.242140 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d80a2b0-01e8-47ce-ae6c-fde41d7545c5 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:55.242958 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d80a2b0-01e8-47ce-ae6c-fde41d7545c5 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:55.243235 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d80a2b0-01e8-47ce-ae6c-fde41d7545c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:55.244844 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3d80a2b0-01e8-47ce-ae6c-fde41d7545c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:55.357582 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d80a2b0-01e8-47ce-ae6c-fde41d7545c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:29:55.359187 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d80a2b0-01e8-47ce-ae6c-fde41d7545c5 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:29:55.378067 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 531/1077] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:29:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 09:29:57.599378 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e583a-ffd0-417e-b917-724011e75711 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:57.599943 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e583a-ffd0-417e-b917-724011e75711 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:57.600032 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e583a-ffd0-417e-b917-724011e75711 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:57.601346 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 547/1078] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:29:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:29:57.607368 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2650453e-673e-42ed-ba24-eca7db444f33 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:57.608352 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2650453e-673e-42ed-ba24-eca7db444f33 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:57.608869 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2650453e-673e-42ed-ba24-eca7db444f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:57.652280 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:29:57.689218 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-2650453e-673e-42ed-ba24-eca7db444f33 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:29:57.690552 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2650453e-673e-42ed-ba24-eca7db444f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:57.797401 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 532/1079] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:29:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:29:58.118910 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309e583a-ffd0-417e-b917-724011e75711 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:29:58.120408 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-309e583a-ffd0-417e-b917-724011e75711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:58.190847 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:29:58.293535 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309e583a-ffd0-417e-b917-724011e75711 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:29:58.418029 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30f735b-e5c0-4bcb-bfb6-3e1d27a19c77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe4a6e4b6184826b30ac4fc13ad9b1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:29:58.418662 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f735b-e5c0-4bcb-bfb6-3e1d27a19c77 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:29:58.418937 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f735b-e5c0-4bcb-bfb6-3e1d27a19c77 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:29:58.419199 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f735b-e5c0-4bcb-bfb6-3e1d27a19c77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:29:58.420805 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c30f735b-e5c0-4bcb-bfb6-3e1d27a19c77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:29:58.443029 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:29:58.533262 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30f735b-e5c0-4bcb-bfb6-3e1d27a19c77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:29:58.534852 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30f735b-e5c0-4bcb-bfb6-3e1d27a19c77 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:29:58.556360 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 548/1080] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:29:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:30:00.766872 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c100f2-626b-4bb6-a92b-22d00dc4eed5 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:00.767451 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c100f2-626b-4bb6-a92b-22d00dc4eed5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:00.767451 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c100f2-626b-4bb6-a92b-22d00dc4eed5 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:00.768895 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 533/1081] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:30:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:30:00.774109 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cba4f3-dc6f-4e08-8898-b7f11c08f587 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:00.774523 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cba4f3-dc6f-4e08-8898-b7f11c08f587 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:00.774707 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cba4f3-dc6f-4e08-8898-b7f11c08f587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:00.844298 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-b3cba4f3-dc6f-4e08-8898-b7f11c08f587 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:30:00.845533 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b3cba4f3-dc6f-4e08-8898-b7f11c08f587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:00.945003 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:30:00.951739 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 549/1082] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:30:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:30:01.245298 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c100f2-626b-4bb6-a92b-22d00dc4eed5 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:30:01.246816 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-21c100f2-626b-4bb6-a92b-22d00dc4eed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:01.370275 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:30:01.430169 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c100f2-626b-4bb6-a92b-22d00dc4eed5 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:30:01.558018 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ff958-ad18-4854-acbe-2f7fd25717ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe4a6e4b6184826b30ac4fc13ad9b1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:30:01.558713 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ff958-ad18-4854-acbe-2f7fd25717ee service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:01.559058 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ff958-ad18-4854-acbe-2f7fd25717ee service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:01.559396 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ff958-ad18-4854-acbe-2f7fd25717ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:01.561077 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-302ff958-ad18-4854-acbe-2f7fd25717ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:01.642116 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:30:01.665667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302ff958-ad18-4854-acbe-2f7fd25717ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:30:01.667275 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302ff958-ad18-4854-acbe-2f7fd25717ee service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:30:01.688546 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 534/1083] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:30:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:30:03.972502 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb77b14-b456-4aaf-b8e1-700da4fab609 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:03.973056 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb77b14-b456-4aaf-b8e1-700da4fab609 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:03.973056 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb77b14-b456-4aaf-b8e1-700da4fab609 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:03.974398 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 550/1084] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:30:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:30:03.979748 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd3275-a66b-42be-942f-c956205ae7ee None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:03.980175 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd3275-a66b-42be-942f-c956205ae7ee None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:03.980357 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd3275-a66b-42be-942f-c956205ae7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:04.054265 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b7bd3275-a66b-42be-942f-c956205ae7ee None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:30:04.055625 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b7bd3275-a66b-42be-942f-c956205ae7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:04.166449 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 535/1085] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:30:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:30:04.462433 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb77b14-b456-4aaf-b8e1-700da4fab609 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:30:04.464456 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9eb77b14-b456-4aaf-b8e1-700da4fab609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:04.630498 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb77b14-b456-4aaf-b8e1-700da4fab609 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:30:04.704589 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:30:04.758385 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9099ea0a-1980-47ef-953d-1fde4fdf622b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe4a6e4b6184826b30ac4fc13ad9b1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:30:04.758993 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9099ea0a-1980-47ef-953d-1fde4fdf622b service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:04.759271 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9099ea0a-1980-47ef-953d-1fde4fdf622b service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:04.759543 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9099ea0a-1980-47ef-953d-1fde4fdf622b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:04.761166 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9099ea0a-1980-47ef-953d-1fde4fdf622b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:04.860787 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9099ea0a-1980-47ef-953d-1fde4fdf622b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:30:04.863634 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9099ea0a-1980-47ef-953d-1fde4fdf622b service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:30:04.883322 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 551/1086] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:30:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:30:14.164464 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=79 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:30:14.880005 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=64 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:30:18.638971 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cfdfa7-338c-4e34-a1c7-00673937c76f None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:18.639546 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cfdfa7-338c-4e34-a1c7-00673937c76f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:18.639546 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cfdfa7-338c-4e34-a1c7-00673937c76f None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:18.640864 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 536/1087] 10.222.0.70 () {54 vars in 918 bytes} [Thu Oct 27 09:30:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:30:18.645863 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de784672-5a4d-4b8b-bb2b-b01403ccba2d None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:18.646324 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de784672-5a4d-4b8b-bb2b-b01403ccba2d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:18.646432 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de784672-5a4d-4b8b-bb2b-b01403ccba2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:18.721758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-de784672-5a4d-4b8b-bb2b-b01403ccba2d None None] MFA Rules not processed for user `2316b30a6e7b4175bd7b90583584eab5`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:30:18.722978 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-de784672-5a4d-4b8b-bb2b-b01403ccba2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:18.827911 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 552/1088] 10.222.0.70 () {58 vars in 1015 bytes} [Thu Oct 27 09:30:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:30:18.842866 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db0624-58be-4c41-8160-d14e8d075279 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:18.843147 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db0624-58be-4c41-8160-d14e8d075279 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:18.843417 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db0624-58be-4c41-8160-d14e8d075279 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:18.844769 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 537/1089] 10.222.0.70 () {54 vars in 918 bytes} [Thu Oct 27 09:30:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:30:18.849264 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a262aa3d-6833-4172-84e8-7aa917840584 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:18.849548 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a262aa3d-6833-4172-84e8-7aa917840584 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:18.849820 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a262aa3d-6833-4172-84e8-7aa917840584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:18.921641 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:30:18.924594 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-a262aa3d-6833-4172-84e8-7aa917840584 None None] MFA Rules not processed for user `dbe4a6e4b6184826b30ac4fc13ad9b1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:30:18.926438 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a262aa3d-6833-4172-84e8-7aa917840584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:19.042540 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 553/1090] 10.222.0.70 () {58 vars in 1015 bytes} [Thu Oct 27 09:30:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:30:21.604541 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e199626-5119-495b-83d1-d039e2f325d3 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:21.605080 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e199626-5119-495b-83d1-d039e2f325d3 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:21.605080 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e199626-5119-495b-83d1-d039e2f325d3 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:21.606453 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 538/1091] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 27 09:30:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:30:21.611810 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fecf39-c1af-4a5a-b142-e43f23052ea7 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:21.612230 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fecf39-c1af-4a5a-b142-e43f23052ea7 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:21.612425 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fecf39-c1af-4a5a-b142-e43f23052ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:21.679672 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-26fecf39-c1af-4a5a-b142-e43f23052ea7 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:30:21.680917 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-26fecf39-c1af-4a5a-b142-e43f23052ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:21.735836 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:30:21.778140 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 554/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 27 09:30:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:30:22.071209 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e199626-5119-495b-83d1-d039e2f325d3 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:30:22.072817 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5e199626-5119-495b-83d1-d039e2f325d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:22.216954 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e199626-5119-495b-83d1-d039e2f325d3 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:30:22.290815 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:30:22.330971 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614cd691-a3f0-478f-9f6a-de0cc5cb1265 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe4a6e4b6184826b30ac4fc13ad9b1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:30:22.331590 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614cd691-a3f0-478f-9f6a-de0cc5cb1265 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:22.331855 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614cd691-a3f0-478f-9f6a-de0cc5cb1265 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:22.332123 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614cd691-a3f0-478f-9f6a-de0cc5cb1265 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:22.333733 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-614cd691-a3f0-478f-9f6a-de0cc5cb1265 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:22.424943 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614cd691-a3f0-478f-9f6a-de0cc5cb1265 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:30:22.426540 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614cd691-a3f0-478f-9f6a-de0cc5cb1265 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:30:22.445264 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 539/1093] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:30:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:30:23.564079 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e868bf4-7d2e-459f-9c94-d742b9ec5a52 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:23.564657 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e868bf4-7d2e-459f-9c94-d742b9ec5a52 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:23.564657 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e868bf4-7d2e-459f-9c94-d742b9ec5a52 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:23.565892 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 555/1094] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 27 09:30:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:30:23.570700 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab0f21-ea62-4c05-8917-afeedca1af7f None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:23.571106 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab0f21-ea62-4c05-8917-afeedca1af7f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:23.571309 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab0f21-ea62-4c05-8917-afeedca1af7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:23.646070 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-39ab0f21-ea62-4c05-8917-afeedca1af7f None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:30:23.647839 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-39ab0f21-ea62-4c05-8917-afeedca1af7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:23.674440 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:30:23.744695 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 540/1095] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 27 09:30:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:30:23.907822 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e868bf4-7d2e-459f-9c94-d742b9ec5a52 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:30:23.909407 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7e868bf4-7d2e-459f-9c94-d742b9ec5a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:24.053144 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e868bf4-7d2e-459f-9c94-d742b9ec5a52 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:30:24.092638 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:30:24.165049 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe36268-105d-4c27-821d-b1ce028a829a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe4a6e4b6184826b30ac4fc13ad9b1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:30:24.165577 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe36268-105d-4c27-821d-b1ce028a829a service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:24.165877 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe36268-105d-4c27-821d-b1ce028a829a service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:24.166144 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe36268-105d-4c27-821d-b1ce028a829a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:24.167694 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9fe36268-105d-4c27-821d-b1ce028a829a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:24.256994 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe36268-105d-4c27-821d-b1ce028a829a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:30:24.258567 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe36268-105d-4c27-821d-b1ce028a829a service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:30:24.276427 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 556/1096] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:30:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:30:25.395796 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3c9c65-d388-4cec-b2df-f4f42f6b06c6 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:25.396411 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3c9c65-d388-4cec-b2df-f4f42f6b06c6 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:25.396411 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3c9c65-d388-4cec-b2df-f4f42f6b06c6 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:25.398294 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 541/1097] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 27 09:30:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:30:25.403468 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979e626-7bcc-4efd-849f-241e26969af4 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:25.403884 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979e626-7bcc-4efd-849f-241e26969af4 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:25.403994 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979e626-7bcc-4efd-849f-241e26969af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:25.457356 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:30:25.480486 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-8979e626-7bcc-4efd-849f-241e26969af4 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:30:25.481714 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8979e626-7bcc-4efd-849f-241e26969af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:25.576360 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 557/1098] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 27 09:30:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:30:25.728801 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3c9c65-d388-4cec-b2df-f4f42f6b06c6 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:30:25.729861 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6d3c9c65-d388-4cec-b2df-f4f42f6b06c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:25.871831 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3c9c65-d388-4cec-b2df-f4f42f6b06c6 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:30:25.883558 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:30:25.984790 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cec260-fd7c-404b-97b5-a980eb72855e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe4a6e4b6184826b30ac4fc13ad9b1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:30:25.985390 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cec260-fd7c-404b-97b5-a980eb72855e service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:25.985668 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cec260-fd7c-404b-97b5-a980eb72855e service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:25.985963 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cec260-fd7c-404b-97b5-a980eb72855e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:25.987567 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-13cec260-fd7c-404b-97b5-a980eb72855e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:26.075380 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cec260-fd7c-404b-97b5-a980eb72855e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:30:26.077074 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cec260-fd7c-404b-97b5-a980eb72855e service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:30:26.095212 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 542/1099] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:30:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:30:27.186239 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7879ab02-c4d0-4fc9-bd81-38932f30e2f1 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:27.186753 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7879ab02-c4d0-4fc9-bd81-38932f30e2f1 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:27.186753 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7879ab02-c4d0-4fc9-bd81-38932f30e2f1 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:27.188176 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 558/1100] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 27 09:30:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:30:27.193459 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb397e1-37fc-4a17-b89e-9100b5143a1b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:27.193930 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb397e1-37fc-4a17-b89e-9100b5143a1b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:27.194049 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb397e1-37fc-4a17-b89e-9100b5143a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:27.216217 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:30:27.278998 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-adb397e1-37fc-4a17-b89e-9100b5143a1b None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:30:27.280215 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-adb397e1-37fc-4a17-b89e-9100b5143a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:27.380899 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 543/1101] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 27 09:30:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:30:27.531375 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7879ab02-c4d0-4fc9-bd81-38932f30e2f1 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:30:27.532844 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7879ab02-c4d0-4fc9-bd81-38932f30e2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:27.646414 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:30:27.683895 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7879ab02-c4d0-4fc9-bd81-38932f30e2f1 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:30:27.799774 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079a0eb1-5650-4495-9148-177b4cefd7fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe4a6e4b6184826b30ac4fc13ad9b1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:30:27.800392 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079a0eb1-5650-4495-9148-177b4cefd7fe service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:27.800711 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079a0eb1-5650-4495-9148-177b4cefd7fe service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:27.800981 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079a0eb1-5650-4495-9148-177b4cefd7fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:27.802562 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-079a0eb1-5650-4495-9148-177b4cefd7fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:27.884828 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:30:27.899683 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079a0eb1-5650-4495-9148-177b4cefd7fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:30:27.901385 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079a0eb1-5650-4495-9148-177b4cefd7fe service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:30:27.919692 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 559/1102] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:30:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:30:29.082956 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b4b33-a96d-4a60-876c-3440a22ab86e None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:29.083498 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b4b33-a96d-4a60-876c-3440a22ab86e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:29.083498 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b4b33-a96d-4a60-876c-3440a22ab86e None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:29.084882 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 544/1103] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 27 09:30:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:30:29.090283 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be18bc7-9e31-462a-8ff1-44410a6ba070 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:29.090754 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be18bc7-9e31-462a-8ff1-44410a6ba070 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:29.090847 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be18bc7-9e31-462a-8ff1-44410a6ba070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:29.170912 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-0be18bc7-9e31-462a-8ff1-44410a6ba070 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:30:29.172181 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0be18bc7-9e31-462a-8ff1-44410a6ba070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:29.275226 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 560/1104] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 27 09:30:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:30:29.426711 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b4b33-a96d-4a60-876c-3440a22ab86e None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:30:29.428245 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-bc0b4b33-a96d-4a60-876c-3440a22ab86e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:29.519067 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:30:29.581141 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b4b33-a96d-4a60-876c-3440a22ab86e None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:30:29.702607 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c28fe1c-39a5-41e9-8fac-294f7380fd5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe4a6e4b6184826b30ac4fc13ad9b1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:30:29.703210 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c28fe1c-39a5-41e9-8fac-294f7380fd5a service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:29.703469 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c28fe1c-39a5-41e9-8fac-294f7380fd5a service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:29.703741 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c28fe1c-39a5-41e9-8fac-294f7380fd5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:29.705368 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5c28fe1c-39a5-41e9-8fac-294f7380fd5a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:29.757718 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:30:29.801126 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c28fe1c-39a5-41e9-8fac-294f7380fd5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:30:29.802726 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c28fe1c-39a5-41e9-8fac-294f7380fd5a service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:30:29.821650 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 545/1105] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:30:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:30:30.889841 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a62d1-0ed2-43ee-bb4e-1a3c1b1ba3f3 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:30.890395 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a62d1-0ed2-43ee-bb4e-1a3c1b1ba3f3 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:30.890395 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a62d1-0ed2-43ee-bb4e-1a3c1b1ba3f3 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:30.891734 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 561/1106] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 27 09:30:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:30:30.896964 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a0e6c-502e-409a-8465-ed4adb31dd93 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:30.897432 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a0e6c-502e-409a-8465-ed4adb31dd93 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:30.897533 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a0e6c-502e-409a-8465-ed4adb31dd93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:30.977971 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b96a0e6c-502e-409a-8465-ed4adb31dd93 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:30:30.979134 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b96a0e6c-502e-409a-8465-ed4adb31dd93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:31.081977 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 546/1107] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 27 09:30:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:30:31.236006 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733a62d1-0ed2-43ee-bb4e-1a3c1b1ba3f3 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:30:31.237655 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-733a62d1-0ed2-43ee-bb4e-1a3c1b1ba3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:31.294077 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:30:31.396718 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733a62d1-0ed2-43ee-bb4e-1a3c1b1ba3f3 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:30:31.515098 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341dea5-34ea-4278-8d02-fe8937c779b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe4a6e4b6184826b30ac4fc13ad9b1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:30:31.515706 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341dea5-34ea-4278-8d02-fe8937c779b9 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:31.515981 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341dea5-34ea-4278-8d02-fe8937c779b9 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:31.516249 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341dea5-34ea-4278-8d02-fe8937c779b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:31.517922 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1341dea5-34ea-4278-8d02-fe8937c779b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:31.525521 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:30:31.613169 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1341dea5-34ea-4278-8d02-fe8937c779b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:30:31.614767 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1341dea5-34ea-4278-8d02-fe8937c779b9 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:30:31.634078 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 562/1108] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:30:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:30:32.689578 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc57ee8-4775-4986-a827-b7fe92b32d3a None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:32.690114 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc57ee8-4775-4986-a827-b7fe92b32d3a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:32.690114 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc57ee8-4775-4986-a827-b7fe92b32d3a None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:32.691455 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 547/1109] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 27 09:30:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:30:32.696470 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ba977-e628-43f4-a080-95d5e3c50991 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:32.696920 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ba977-e628-43f4-a080-95d5e3c50991 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:32.697050 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ba977-e628-43f4-a080-95d5e3c50991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:32.777012 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-445ba977-e628-43f4-a080-95d5e3c50991 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:30:32.778213 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-445ba977-e628-43f4-a080-95d5e3c50991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:32.847435 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:30:32.878098 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 563/1110] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 27 09:30:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:30:33.029807 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc57ee8-4775-4986-a827-b7fe92b32d3a None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:30:33.031229 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2fc57ee8-4775-4986-a827-b7fe92b32d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:33.049022 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:30:33.186544 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc57ee8-4775-4986-a827-b7fe92b32d3a None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:30:33.283230 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:30:33.311335 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efebf749-3671-4bde-a14a-4dac18d5bc8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe4a6e4b6184826b30ac4fc13ad9b1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:30:33.311939 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efebf749-3671-4bde-a14a-4dac18d5bc8d service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:33.312238 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efebf749-3671-4bde-a14a-4dac18d5bc8d service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:33.312508 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efebf749-3671-4bde-a14a-4dac18d5bc8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:33.314110 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-efebf749-3671-4bde-a14a-4dac18d5bc8d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:33.407555 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efebf749-3671-4bde-a14a-4dac18d5bc8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:30:33.409261 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efebf749-3671-4bde-a14a-4dac18d5bc8d service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:30:33.427177 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 548/1111] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:30:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:30:34.467933 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc879792-53bc-4ee4-9f73-423c61c54ae2 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:34.468364 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc879792-53bc-4ee4-9f73-423c61c54ae2 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:34.468491 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc879792-53bc-4ee4-9f73-423c61c54ae2 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:34.469862 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 564/1112] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 27 09:30:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:30:34.474804 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91d512-14a4-4257-a37f-6d43bb393de6 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:34.475241 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91d512-14a4-4257-a37f-6d43bb393de6 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:34.475338 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91d512-14a4-4257-a37f-6d43bb393de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:34.554130 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-fb91d512-14a4-4257-a37f-6d43bb393de6 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:30:34.555381 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fb91d512-14a4-4257-a37f-6d43bb393de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:34.598202 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:30:34.653238 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 549/1113] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 27 09:30:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:30:34.802579 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc879792-53bc-4ee4-9f73-423c61c54ae2 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:30:34.804793 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-dc879792-53bc-4ee4-9f73-423c61c54ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:34.946841 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc879792-53bc-4ee4-9f73-423c61c54ae2 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:30:35.001476 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:30:35.059456 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9278227-af3b-423d-b6b1-70afacd17c25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe4a6e4b6184826b30ac4fc13ad9b1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:30:35.059994 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9278227-af3b-423d-b6b1-70afacd17c25 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:35.060271 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9278227-af3b-423d-b6b1-70afacd17c25 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:35.060570 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9278227-af3b-423d-b6b1-70afacd17c25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:35.062150 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b9278227-af3b-423d-b6b1-70afacd17c25 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:35.151090 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9278227-af3b-423d-b6b1-70afacd17c25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:30:35.152703 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9278227-af3b-423d-b6b1-70afacd17c25 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:30:35.170200 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 565/1114] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:30:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:30:36.217933 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696285ae-e97a-4857-9e03-59e9c3e37e0b None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:36.218509 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696285ae-e97a-4857-9e03-59e9c3e37e0b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:36.218509 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696285ae-e97a-4857-9e03-59e9c3e37e0b None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:36.219825 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 550/1115] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 27 09:30:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:30:36.225339 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705bcf3f-022c-4334-bb5e-622a80d17518 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:36.225770 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705bcf3f-022c-4334-bb5e-622a80d17518 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:36.225908 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705bcf3f-022c-4334-bb5e-622a80d17518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:36.300280 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-705bcf3f-022c-4334-bb5e-622a80d17518 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:30:36.301537 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-705bcf3f-022c-4334-bb5e-622a80d17518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:36.304188 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:30:36.399225 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 566/1116] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 27 09:30:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:30:36.549505 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696285ae-e97a-4857-9e03-59e9c3e37e0b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:30:36.550923 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-696285ae-e97a-4857-9e03-59e9c3e37e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:36.694201 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696285ae-e97a-4857-9e03-59e9c3e37e0b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:30:36.720709 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:30:36.807099 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf2fc11-6092-4fe3-9b1c-b9fd161f1f13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe4a6e4b6184826b30ac4fc13ad9b1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:30:36.807618 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf2fc11-6092-4fe3-9b1c-b9fd161f1f13 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:36.807894 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf2fc11-6092-4fe3-9b1c-b9fd161f1f13 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:36.808160 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf2fc11-6092-4fe3-9b1c-b9fd161f1f13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:36.809785 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9cf2fc11-6092-4fe3-9b1c-b9fd161f1f13 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:36.898858 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf2fc11-6092-4fe3-9b1c-b9fd161f1f13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:30:36.900465 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf2fc11-6092-4fe3-9b1c-b9fd161f1f13 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:30:36.918101 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 551/1117] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:30:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:30:38.966988 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87edde-6541-4256-bebc-8fa77c175523 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:38.967640 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87edde-6541-4256-bebc-8fa77c175523 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:38.967640 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87edde-6541-4256-bebc-8fa77c175523 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:38.968950 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 567/1118] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 27 09:30:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:30:38.974187 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ddacd0-6f5f-411c-9430-a2b5d33184b1 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:38.974654 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ddacd0-6f5f-411c-9430-a2b5d33184b1 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:38.974791 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ddacd0-6f5f-411c-9430-a2b5d33184b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:39.016479 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:30:39.042596 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-e0ddacd0-6f5f-411c-9430-a2b5d33184b1 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:30:39.043737 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e0ddacd0-6f5f-411c-9430-a2b5d33184b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:39.140658 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 552/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 27 09:30:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:30:39.431935 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d87edde-6541-4256-bebc-8fa77c175523 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:30:39.433517 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1d87edde-6541-4256-bebc-8fa77c175523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:39.534559 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d87edde-6541-4256-bebc-8fa77c175523 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:30:39.566111 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:30:39.607620 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb30970-14c8-43be-90bf-25cb75124e90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:30:39.608216 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb30970-14c8-43be-90bf-25cb75124e90 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:39.608493 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb30970-14c8-43be-90bf-25cb75124e90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:39.608794 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb30970-14c8-43be-90bf-25cb75124e90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:39.610352 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-efb30970-14c8-43be-90bf-25cb75124e90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:39.719352 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb30970-14c8-43be-90bf-25cb75124e90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:30:39.720731 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb30970-14c8-43be-90bf-25cb75124e90 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:30:39.738216 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 568/1120] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:30:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:30:42.044242 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3963c1a2-a026-40bc-87ab-8dab9c71217a None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:42.044841 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3963c1a2-a026-40bc-87ab-8dab9c71217a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:42.044841 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3963c1a2-a026-40bc-87ab-8dab9c71217a None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:42.046161 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 553/1121] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 27 09:30:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:30:42.051504 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1c922-d5cc-4605-90f3-163ca30b78d5 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:42.051935 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1c922-d5cc-4605-90f3-163ca30b78d5 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:42.052058 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1c922-d5cc-4605-90f3-163ca30b78d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:42.117907 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-0aa1c922-d5cc-4605-90f3-163ca30b78d5 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:30:42.119111 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0aa1c922-d5cc-4605-90f3-163ca30b78d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:42.131583 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:30:42.214399 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 569/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 27 09:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:30:42.499850 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3963c1a2-a026-40bc-87ab-8dab9c71217a None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:30:42.501423 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3963c1a2-a026-40bc-87ab-8dab9c71217a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:42.604389 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3963c1a2-a026-40bc-87ab-8dab9c71217a None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:30:42.616908 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:30:42.678053 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec6b9d-610e-486e-ac47-bf5cf35e97f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:30:42.678647 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec6b9d-610e-486e-ac47-bf5cf35e97f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:42.678922 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec6b9d-610e-486e-ac47-bf5cf35e97f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:42.679190 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec6b9d-610e-486e-ac47-bf5cf35e97f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:42.680790 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-49ec6b9d-610e-486e-ac47-bf5cf35e97f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:42.788714 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ec6b9d-610e-486e-ac47-bf5cf35e97f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:30:42.790105 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ec6b9d-610e-486e-ac47-bf5cf35e97f8 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:30:42.808775 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 554/1123] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:30:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:30:45.254141 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973acad-7736-4ece-95d2-9d37c1d14443 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:45.254608 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973acad-7736-4ece-95d2-9d37c1d14443 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:45.254726 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973acad-7736-4ece-95d2-9d37c1d14443 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:45.256054 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 570/1124] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 27 09:30:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:30:45.261486 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9f488-11d0-47a3-aab2-1cda7e539e4d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:45.261891 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9f488-11d0-47a3-aab2-1cda7e539e4d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:45.262028 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9f488-11d0-47a3-aab2-1cda7e539e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:45.312010 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:30:45.330050 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-21d9f488-11d0-47a3-aab2-1cda7e539e4d None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:30:45.331254 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-21d9f488-11d0-47a3-aab2-1cda7e539e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:45.429575 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 555/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 27 09:30:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:30:45.714322 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973acad-7736-4ece-95d2-9d37c1d14443 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:30:45.715885 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2973acad-7736-4ece-95d2-9d37c1d14443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:45.816403 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973acad-7736-4ece-95d2-9d37c1d14443 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:30:45.856370 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:30:45.889525 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9456c6-4744-4e56-945b-30b583dc68d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:30:45.890119 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9456c6-4744-4e56-945b-30b583dc68d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:45.890389 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9456c6-4744-4e56-945b-30b583dc68d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:45.890650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9456c6-4744-4e56-945b-30b583dc68d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:45.892205 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9f9456c6-4744-4e56-945b-30b583dc68d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:46.000427 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9456c6-4744-4e56-945b-30b583dc68d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:30:46.001808 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9456c6-4744-4e56-945b-30b583dc68d0 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:30:46.019222 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 571/1126] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:30:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:30:48.292888 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c79b7-394f-4566-97b5-78fb59963ce6 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:48.293348 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c79b7-394f-4566-97b5-78fb59963ce6 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:48.293444 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c79b7-394f-4566-97b5-78fb59963ce6 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:48.294826 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 556/1127] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 27 09:30:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:30:48.299961 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e21d8ba-f170-4df9-a7cc-019c486e4df7 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:48.300397 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e21d8ba-f170-4df9-a7cc-019c486e4df7 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:48.300503 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e21d8ba-f170-4df9-a7cc-019c486e4df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:48.367325 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-1e21d8ba-f170-4df9-a7cc-019c486e4df7 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:30:48.368549 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1e21d8ba-f170-4df9-a7cc-019c486e4df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:48.388725 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:30:48.463804 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 572/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 27 09:30:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:30:48.748416 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c79b7-394f-4566-97b5-78fb59963ce6 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:30:48.749975 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-cd2c79b7-394f-4566-97b5-78fb59963ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:48.854767 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c79b7-394f-4566-97b5-78fb59963ce6 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:30:48.875340 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:30:48.929493 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442d1f1d-cebd-4cbd-9980-c82ea9824525 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:30:48.930097 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442d1f1d-cebd-4cbd-9980-c82ea9824525 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:48.930376 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442d1f1d-cebd-4cbd-9980-c82ea9824525 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:48.930646 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442d1f1d-cebd-4cbd-9980-c82ea9824525 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:48.932230 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-442d1f1d-cebd-4cbd-9980-c82ea9824525 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:49.040381 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442d1f1d-cebd-4cbd-9980-c82ea9824525 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:30:49.041758 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442d1f1d-cebd-4cbd-9980-c82ea9824525 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:30:49.059674 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 557/1129] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:30:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:30:51.262148 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a5c927-f709-4f95-9c50-e4e094f235b9 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:51.262561 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a5c927-f709-4f95-9c50-e4e094f235b9 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:51.262692 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a5c927-f709-4f95-9c50-e4e094f235b9 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:51.264035 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 573/1130] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 27 09:30:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:30:51.269334 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7733b9f-b242-4d20-bcf8-280677c05553 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:51.269803 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7733b9f-b242-4d20-bcf8-280677c05553 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:51.269911 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7733b9f-b242-4d20-bcf8-280677c05553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:51.332813 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:30:51.336743 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a7733b9f-b242-4d20-bcf8-280677c05553 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:30:51.338752 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a7733b9f-b242-4d20-bcf8-280677c05553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:51.435862 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 558/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 27 09:30:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:30:51.719244 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a5c927-f709-4f95-9c50-e4e094f235b9 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:30:51.720801 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b3a5c927-f709-4f95-9c50-e4e094f235b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:51.822544 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a5c927-f709-4f95-9c50-e4e094f235b9 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:30:51.872214 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:30:51.897058 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1dfbe7-53c6-4987-8f67-6e2891bc68db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:30:51.897655 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1dfbe7-53c6-4987-8f67-6e2891bc68db service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:51.897933 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1dfbe7-53c6-4987-8f67-6e2891bc68db service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:51.898202 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1dfbe7-53c6-4987-8f67-6e2891bc68db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:51.899771 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6e1dfbe7-53c6-4987-8f67-6e2891bc68db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:52.006255 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1dfbe7-53c6-4987-8f67-6e2891bc68db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:30:52.007591 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1dfbe7-53c6-4987-8f67-6e2891bc68db service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:30:52.026298 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 574/1132] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:30:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:30:54.240826 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de33eca9-4428-4176-b773-fe3065cc5ad0 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:54.241395 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de33eca9-4428-4176-b773-fe3065cc5ad0 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:54.241659 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de33eca9-4428-4176-b773-fe3065cc5ad0 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:54.243011 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 559/1133] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 27 09:30:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:30:54.248552 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47db73c9-9f79-46c2-abbf-6fd918a21b0d None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:54.248963 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47db73c9-9f79-46c2-abbf-6fd918a21b0d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:54.249106 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47db73c9-9f79-46c2-abbf-6fd918a21b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:54.315245 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-47db73c9-9f79-46c2-abbf-6fd918a21b0d None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:30:54.316451 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-47db73c9-9f79-46c2-abbf-6fd918a21b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:54.345301 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:30:54.413396 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 575/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 27 09:30:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:30:54.695041 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de33eca9-4428-4176-b773-fe3065cc5ad0 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:30:54.696544 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-de33eca9-4428-4176-b773-fe3065cc5ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:54.798408 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de33eca9-4428-4176-b773-fe3065cc5ad0 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:30:54.826768 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:30:54.873696 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeec263-fe5b-46f9-9a6c-fe7c00b456db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:30:54.874293 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeec263-fe5b-46f9-9a6c-fe7c00b456db service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:54.874566 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeec263-fe5b-46f9-9a6c-fe7c00b456db service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:54.874830 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeec263-fe5b-46f9-9a6c-fe7c00b456db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:54.876402 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-eaeec263-fe5b-46f9-9a6c-fe7c00b456db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:54.983751 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeec263-fe5b-46f9-9a6c-fe7c00b456db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:30:54.985141 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeec263-fe5b-46f9-9a6c-fe7c00b456db service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:30:55.003152 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 560/1135] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:30:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:30:57.200294 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6fb7fa-8052-4367-b047-d700f739879b None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:57.200906 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6fb7fa-8052-4367-b047-d700f739879b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:57.200906 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6fb7fa-8052-4367-b047-d700f739879b None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:57.202211 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 576/1136] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 27 09:30:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:30:57.207425 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f4e40b-12a5-4b58-926d-ee0981237fc9 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:57.207853 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f4e40b-12a5-4b58-926d-ee0981237fc9 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:57.207964 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f4e40b-12a5-4b58-926d-ee0981237fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:57.275084 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-d4f4e40b-12a5-4b58-926d-ee0981237fc9 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:30:57.276297 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d4f4e40b-12a5-4b58-926d-ee0981237fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:57.286574 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:30:57.374556 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 561/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 27 09:30:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:30:57.661775 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6fb7fa-8052-4367-b047-d700f739879b None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:30:57.663300 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0e6fb7fa-8052-4367-b047-d700f739879b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:57.764592 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6fb7fa-8052-4367-b047-d700f739879b None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:30:57.819513 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:30:57.838220 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce768ad-b4f0-4f06-9476-f4cd67672154 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:30:57.838819 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce768ad-b4f0-4f06-9476-f4cd67672154 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:30:57.839096 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce768ad-b4f0-4f06-9476-f4cd67672154 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:30:57.839364 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce768ad-b4f0-4f06-9476-f4cd67672154 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:30:57.840962 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1ce768ad-b4f0-4f06-9476-f4cd67672154 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:30:57.949237 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce768ad-b4f0-4f06-9476-f4cd67672154 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:30:57.950598 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce768ad-b4f0-4f06-9476-f4cd67672154 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:30:57.969206 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 577/1138] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:30:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:31:00.137708 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345651b2-c47b-4d64-b87a-99e25a829d61 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:31:00.138253 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345651b2-c47b-4d64-b87a-99e25a829d61 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:31:00.138253 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345651b2-c47b-4d64-b87a-99e25a829d61 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:31:00.139729 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 562/1139] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 27 09:31:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:31:00.144777 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111cf39-bc08-4c7c-ae13-23d0ed94f2c8 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:31:00.145230 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111cf39-bc08-4c7c-ae13-23d0ed94f2c8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:31:00.145324 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111cf39-bc08-4c7c-ae13-23d0ed94f2c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:31:00.211371 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-3111cf39-bc08-4c7c-ae13-23d0ed94f2c8 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:31:00.212865 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3111cf39-bc08-4c7c-ae13-23d0ed94f2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:31:00.251328 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:31:00.310626 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 578/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 27 09:31:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:31:00.590560 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345651b2-c47b-4d64-b87a-99e25a829d61 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:31:00.592079 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-345651b2-c47b-4d64-b87a-99e25a829d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:31:00.700783 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345651b2-c47b-4d64-b87a-99e25a829d61 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:31:00.740287 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:31:00.778290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea66f31-b7cd-4911-b664-96268101d52f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:31:00.778897 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea66f31-b7cd-4911-b664-96268101d52f service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:31:00.779177 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea66f31-b7cd-4911-b664-96268101d52f service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:31:00.779445 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea66f31-b7cd-4911-b664-96268101d52f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:31:00.781069 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3ea66f31-b7cd-4911-b664-96268101d52f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:31:00.893309 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea66f31-b7cd-4911-b664-96268101d52f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:31:00.894647 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea66f31-b7cd-4911-b664-96268101d52f service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:31:00.912508 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 563/1141] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:31:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:31:03.054686 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f2a068-c187-44bb-823b-3a3f84aed1f9 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:31:03.055218 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f2a068-c187-44bb-823b-3a3f84aed1f9 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:31:03.055218 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f2a068-c187-44bb-823b-3a3f84aed1f9 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:31:03.056506 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 579/1142] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 27 09:31:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:31:03.062018 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ffd740-50b3-4c6d-a4d9-920a16e82864 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:31:03.062462 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ffd740-50b3-4c6d-a4d9-920a16e82864 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:31:03.062560 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ffd740-50b3-4c6d-a4d9-920a16e82864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:31:03.127631 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-f5ffd740-50b3-4c6d-a4d9-920a16e82864 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:31:03.128842 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f5ffd740-50b3-4c6d-a4d9-920a16e82864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:31:03.146284 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:31:03.222983 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 564/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 27 09:31:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:31:03.506255 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f2a068-c187-44bb-823b-3a3f84aed1f9 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:31:03.507774 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-22f2a068-c187-44bb-823b-3a3f84aed1f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:31:03.611447 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f2a068-c187-44bb-823b-3a3f84aed1f9 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:31:03.678701 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:31:03.685421 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a1ee33-f227-4daa-b78b-24fb1fcea269 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:31:03.686052 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a1ee33-f227-4daa-b78b-24fb1fcea269 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:31:03.686324 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a1ee33-f227-4daa-b78b-24fb1fcea269 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:31:03.686581 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a1ee33-f227-4daa-b78b-24fb1fcea269 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:31:03.688132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a5a1ee33-f227-4daa-b78b-24fb1fcea269 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:31:03.796787 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a1ee33-f227-4daa-b78b-24fb1fcea269 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:31:03.798139 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a1ee33-f227-4daa-b78b-24fb1fcea269 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:31:03.816245 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 580/1144] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:31:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:31:05.989915 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ba4ea-b1ea-4350-9ad8-29ac983137b2 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:31:05.990484 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ba4ea-b1ea-4350-9ad8-29ac983137b2 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:31:05.990484 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ba4ea-b1ea-4350-9ad8-29ac983137b2 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:31:05.991797 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 565/1145] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 27 09:31:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:31:05.997095 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4a29f-8e87-4909-aac6-03d851b08360 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:31:05.997560 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4a29f-8e87-4909-aac6-03d851b08360 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:31:05.997707 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4a29f-8e87-4909-aac6-03d851b08360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:31:06.063184 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-55b4a29f-8e87-4909-aac6-03d851b08360 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:31:06.064359 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-55b4a29f-8e87-4909-aac6-03d851b08360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:31:06.111183 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:31:06.158734 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 581/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 27 09:31:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:31:06.446011 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96ba4ea-b1ea-4350-9ad8-29ac983137b2 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:31:06.447518 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e96ba4ea-b1ea-4350-9ad8-29ac983137b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:31:06.549401 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96ba4ea-b1ea-4350-9ad8-29ac983137b2 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:31:06.597201 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:31:06.622935 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac0869-eae9-4b8f-87b3-f842d18f3f0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:31:06.623530 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac0869-eae9-4b8f-87b3-f842d18f3f0e service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:31:06.623808 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac0869-eae9-4b8f-87b3-f842d18f3f0e service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:31:06.624079 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac0869-eae9-4b8f-87b3-f842d18f3f0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:31:06.625698 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-68ac0869-eae9-4b8f-87b3-f842d18f3f0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:31:06.733500 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ac0869-eae9-4b8f-87b3-f842d18f3f0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:31:06.734921 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ac0869-eae9-4b8f-87b3-f842d18f3f0e service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:31:06.753056 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 566/1147] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:31:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:31:16.157251 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=22 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:31:16.750805 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=66 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:31:26.649530 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c0db6f-25b9-4a82-aebb-3cad28d21efc None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:31:26.650106 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c0db6f-25b9-4a82-aebb-3cad28d21efc None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:31:26.650106 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c0db6f-25b9-4a82-aebb-3cad28d21efc None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:31:26.651447 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 582/1148] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:31:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:31:26.656725 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec81b3-2357-40d6-b945-b16944a78dba None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:31:26.657172 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec81b3-2357-40d6-b945-b16944a78dba None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:31:26.657289 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec81b3-2357-40d6-b945-b16944a78dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:31:26.724344 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-49ec81b3-2357-40d6-b945-b16944a78dba None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:31:26.725578 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-49ec81b3-2357-40d6-b945-b16944a78dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:31:26.820429 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 567/1149] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:31:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:31:27.115224 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc54f0d8-6e2a-417c-8b54-a9f373c354f3 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:31:27.115509 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc54f0d8-6e2a-417c-8b54-a9f373c354f3 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:31:27.115781 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc54f0d8-6e2a-417c-8b54-a9f373c354f3 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:31:27.117134 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 583/1150] 10.222.0.70 () {54 vars in 962 bytes} [Thu Oct 27 09:31:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:31:27.122161 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563963da-4bf5-45ce-82b9-83318cbdb5b5 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:31:27.122454 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563963da-4bf5-45ce-82b9-83318cbdb5b5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:31:27.122732 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563963da-4bf5-45ce-82b9-83318cbdb5b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:31:27.198263 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-563963da-4bf5-45ce-82b9-83318cbdb5b5 None None] MFA Rules not processed for user `72b0538f0e42446887b1d95b2fb41fb2`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:31:27.199475 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-563963da-4bf5-45ce-82b9-83318cbdb5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:31:27.220736 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:31:27.275943 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 568/1151] 10.222.0.70 () {58 vars in 1059 bytes} [Thu Oct 27 09:31:27 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:31:27.283744 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc54f0d8-6e2a-417c-8b54-a9f373c354f3 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:31:27.285485 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-fc54f0d8-6e2a-417c-8b54-a9f373c354f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:31:27.386490 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc54f0d8-6e2a-417c-8b54-a9f373c354f3 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:31:27.459542 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fd2efe-d61d-450d-8197-19ee3fba6bca service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72b0538f0e42446887b1d95b2fb41fb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:31:27.460146 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd2efe-d61d-450d-8197-19ee3fba6bca service glance] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:31:27.460422 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd2efe-d61d-450d-8197-19ee3fba6bca service glance] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:31:27.460722 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd2efe-d61d-450d-8197-19ee3fba6bca service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:31:27.462557 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a6fd2efe-d61d-450d-8197-19ee3fba6bca service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:31:27.515950 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:31:27.572291 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fd2efe-d61d-450d-8197-19ee3fba6bca service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:31:27.573764 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fd2efe-d61d-450d-8197-19ee3fba6bca service glance] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:31:27.592314 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 584/1152] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:31:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:31:29.908679 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58532bd-dbf2-4e7d-9289-30d901ea49d5 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:31:29.909309 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58532bd-dbf2-4e7d-9289-30d901ea49d5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:31:29.909309 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58532bd-dbf2-4e7d-9289-30d901ea49d5 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:31:29.910576 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 569/1153] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:31:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:31:29.915967 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a93ead9-4c18-4e33-92b5-f82161b4795a None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:31:29.916372 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a93ead9-4c18-4e33-92b5-f82161b4795a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:31:29.916527 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a93ead9-4c18-4e33-92b5-f82161b4795a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:31:29.983044 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-1a93ead9-4c18-4e33-92b5-f82161b4795a None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:31:29.984233 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1a93ead9-4c18-4e33-92b5-f82161b4795a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:31:30.077593 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 585/1154] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:31:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:31:30.353745 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58532bd-dbf2-4e7d-9289-30d901ea49d5 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:31:30.355272 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d58532bd-dbf2-4e7d-9289-30d901ea49d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:31:30.479681 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58532bd-dbf2-4e7d-9289-30d901ea49d5 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:31:30.529151 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:31:30.572322 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a532092b-9c87-4160-82a2-3c0747a4a9e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:31:30.572834 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a532092b-9c87-4160-82a2-3c0747a4a9e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:31:30.573109 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a532092b-9c87-4160-82a2-3c0747a4a9e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:31:30.573374 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a532092b-9c87-4160-82a2-3c0747a4a9e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:31:30.573997 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a532092b-9c87-4160-82a2-3c0747a4a9e9 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:31:30.575098 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: WARNING py.warnings [None req-a532092b-9c87-4160-82a2-3c0747a4a9e9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 27 09:31:30.575098 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: warnings.warn(msg) Oct 27 09:31:30.575098 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: Oct 27 09:31:30.576227 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a532092b-9c87-4160-82a2-3c0747a4a9e9 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:31:30.595518 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 570/1155] 10.222.0.70 () {56 vars in 1108 bytes} [Thu Oct 27 09:31:30 2022] GET /identity/v3/projects => generated 2553 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:31:32.770581 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3035e3-5933-46e1-b3d7-279cb60ee798 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:31:32.771121 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3035e3-5933-46e1-b3d7-279cb60ee798 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:31:32.771121 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3035e3-5933-46e1-b3d7-279cb60ee798 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:31:32.772618 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 586/1156] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:31:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:31:32.777703 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5bc43d-34d7-4468-a2ef-c1224cdb56c4 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:31:32.778113 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5bc43d-34d7-4468-a2ef-c1224cdb56c4 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:31:32.778242 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5bc43d-34d7-4468-a2ef-c1224cdb56c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:31:32.843663 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-1d5bc43d-34d7-4468-a2ef-c1224cdb56c4 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:31:32.844886 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1d5bc43d-34d7-4468-a2ef-c1224cdb56c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:31:32.939517 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 571/1157] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:31:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:31:33.234670 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3035e3-5933-46e1-b3d7-279cb60ee798 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:31:33.236294 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ca3035e3-5933-46e1-b3d7-279cb60ee798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:31:33.246456 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:31:33.380168 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3035e3-5933-46e1-b3d7-279cb60ee798 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:31:33.459357 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:31:33.492565 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f5a153-90d7-4855-ba9b-eab2663afbe5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe4a6e4b6184826b30ac4fc13ad9b1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:31:33.493132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5a153-90d7-4855-ba9b-eab2663afbe5 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:31:33.493406 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5a153-90d7-4855-ba9b-eab2663afbe5 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:31:33.493676 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5a153-90d7-4855-ba9b-eab2663afbe5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:31:33.495246 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a7f5a153-90d7-4855-ba9b-eab2663afbe5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:31:33.584506 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f5a153-90d7-4855-ba9b-eab2663afbe5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:31:33.586121 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f5a153-90d7-4855-ba9b-eab2663afbe5 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:31:33.604735 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 587/1158] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:31:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:31:35.933449 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5536da6-5152-4f2d-8dca-5294a43b8a6b None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:31:35.934021 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5536da6-5152-4f2d-8dca-5294a43b8a6b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:31:35.934021 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5536da6-5152-4f2d-8dca-5294a43b8a6b None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:31:35.935334 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 572/1159] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:31:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:31:35.941181 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb90dff1-c5d0-4f45-8a35-3cc087ba7ec4 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:31:35.941615 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb90dff1-c5d0-4f45-8a35-3cc087ba7ec4 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:31:35.941768 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb90dff1-c5d0-4f45-8a35-3cc087ba7ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:31:36.009152 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-fb90dff1-c5d0-4f45-8a35-3cc087ba7ec4 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:31:36.010388 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-fb90dff1-c5d0-4f45-8a35-3cc087ba7ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:31:36.056056 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:31:36.106944 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 588/1160] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:31:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:31:36.392022 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5536da6-5152-4f2d-8dca-5294a43b8a6b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:31:36.394222 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d5536da6-5152-4f2d-8dca-5294a43b8a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:31:36.417542 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:31:36.497440 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5536da6-5152-4f2d-8dca-5294a43b8a6b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:31:36.570703 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a6c626-e1de-450a-8333-56eb9ff0dd67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:31:36.571229 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6c626-e1de-450a-8333-56eb9ff0dd67 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:31:36.571506 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6c626-e1de-450a-8333-56eb9ff0dd67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:31:36.571772 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6c626-e1de-450a-8333-56eb9ff0dd67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:31:36.573383 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c1a6c626-e1de-450a-8333-56eb9ff0dd67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:31:36.647247 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:31:36.681681 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a6c626-e1de-450a-8333-56eb9ff0dd67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:31:36.683112 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a6c626-e1de-450a-8333-56eb9ff0dd67 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:31:36.701128 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 573/1161] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:31:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:31:38.888074 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc2e22-f315-404c-9bd9-f738311860d4 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:31:38.888700 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc2e22-f315-404c-9bd9-f738311860d4 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:31:38.888700 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc2e22-f315-404c-9bd9-f738311860d4 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:31:38.890006 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 589/1162] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:31:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:31:38.895499 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5322d7-e502-470f-b721-953305759736 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:31:38.895918 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5322d7-e502-470f-b721-953305759736 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:31:38.896036 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5322d7-e502-470f-b721-953305759736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:31:38.961001 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-cd5322d7-e502-470f-b721-953305759736 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:31:38.962458 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-cd5322d7-e502-470f-b721-953305759736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:31:39.057193 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 574/1163] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:31:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:31:39.338093 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddc2e22-f315-404c-9bd9-f738311860d4 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:31:39.339671 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3ddc2e22-f315-404c-9bd9-f738311860d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:31:39.440706 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddc2e22-f315-404c-9bd9-f738311860d4 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:31:39.510999 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:31:39.512489 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d26a560-9623-4ede-a8ee-03fb8d491b9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:31:39.513118 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d26a560-9623-4ede-a8ee-03fb8d491b9e service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:31:39.513409 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d26a560-9623-4ede-a8ee-03fb8d491b9e service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:31:39.513868 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d26a560-9623-4ede-a8ee-03fb8d491b9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:31:39.515829 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6d26a560-9623-4ede-a8ee-03fb8d491b9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:31:39.623728 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d26a560-9623-4ede-a8ee-03fb8d491b9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:31:39.625172 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d26a560-9623-4ede-a8ee-03fb8d491b9e service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:31:39.645504 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 590/1164] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:31:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:31:41.899376 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8640f86-e6d9-49d7-98c8-8de99767b03e None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:31:41.899975 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8640f86-e6d9-49d7-98c8-8de99767b03e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:31:41.899975 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8640f86-e6d9-49d7-98c8-8de99767b03e None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:31:41.901295 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 575/1165] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:31:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:31:41.906565 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca460d-ad76-449f-8d7c-e44d9101e6ea None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:31:41.907004 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca460d-ad76-449f-8d7c-e44d9101e6ea None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:31:41.907109 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca460d-ad76-449f-8d7c-e44d9101e6ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:31:41.973090 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-caca460d-ad76-449f-8d7c-e44d9101e6ea None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:31:41.974272 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-caca460d-ad76-449f-8d7c-e44d9101e6ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:31:42.025762 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:31:42.070011 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 591/1166] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:31:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:31:42.361927 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8640f86-e6d9-49d7-98c8-8de99767b03e None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:31:42.363465 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e8640f86-e6d9-49d7-98c8-8de99767b03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:31:42.399551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:31:42.466382 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8640f86-e6d9-49d7-98c8-8de99767b03e None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:31:42.537863 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f19fb3-563f-41ee-9ee9-8a67969e14e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:31:42.538394 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f19fb3-563f-41ee-9ee9-8a67969e14e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:31:42.538664 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f19fb3-563f-41ee-9ee9-8a67969e14e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:31:42.538928 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f19fb3-563f-41ee-9ee9-8a67969e14e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:31:42.540475 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-43f19fb3-563f-41ee-9ee9-8a67969e14e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:31:42.618203 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:31:42.647843 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f19fb3-563f-41ee-9ee9-8a67969e14e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:31:42.649321 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f19fb3-563f-41ee-9ee9-8a67969e14e3 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:31:42.667661 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 576/1167] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:31:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:31:45.031753 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f4181-1a1e-4c82-b8e8-995b3e93b387 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:31:45.032335 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f4181-1a1e-4c82-b8e8-995b3e93b387 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:31:45.032335 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f4181-1a1e-4c82-b8e8-995b3e93b387 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:31:45.033654 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 592/1168] 10.222.0.70 () {54 vars in 927 bytes} [Thu Oct 27 09:31:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:31:45.039383 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6684cd2-f270-472c-9fd6-e87806fa1481 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:31:45.039771 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6684cd2-f270-472c-9fd6-e87806fa1481 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:31:45.039927 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6684cd2-f270-472c-9fd6-e87806fa1481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:31:45.105064 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-c6684cd2-f270-472c-9fd6-e87806fa1481 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:31:45.106265 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c6684cd2-f270-472c-9fd6-e87806fa1481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:31:45.202328 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 577/1169] 10.222.0.70 () {58 vars in 1024 bytes} [Thu Oct 27 09:31:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:31:45.491765 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526f4181-1a1e-4c82-b8e8-995b3e93b387 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:31:45.493343 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-526f4181-1a1e-4c82-b8e8-995b3e93b387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:31:45.595263 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526f4181-1a1e-4c82-b8e8-995b3e93b387 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:31:45.672067 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9d4f99-13b1-4ab5-9503-227fb842fb53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:31:45.672715 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d4f99-13b1-4ab5-9503-227fb842fb53 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:31:45.672990 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d4f99-13b1-4ab5-9503-227fb842fb53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:31:45.673258 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d4f99-13b1-4ab5-9503-227fb842fb53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:31:45.674862 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7d9d4f99-13b1-4ab5-9503-227fb842fb53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:31:45.683622 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:31:45.782808 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9d4f99-13b1-4ab5-9503-227fb842fb53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:31:45.784235 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9d4f99-13b1-4ab5-9503-227fb842fb53 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:31:45.801765 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 593/1170] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:31:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:31:55.200840 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=82 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:31:55.799522 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=50 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:32:58.298524 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a2b5b-7685-4758-8b95-9840b3cb77fb None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:32:58.299006 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a2b5b-7685-4758-8b95-9840b3cb77fb None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:32:58.299006 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a2b5b-7685-4758-8b95-9840b3cb77fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:32:58.374242 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-e02a2b5b-7685-4758-8b95-9840b3cb77fb None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:32:58.375992 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e02a2b5b-7685-4758-8b95-9840b3cb77fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:32:58.470387 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 578/1171] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:32:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:32:58.478482 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4fd2ca-f811-4f62-8b19-7818a9150d0d None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:32:58.479039 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4fd2ca-f811-4f62-8b19-7818a9150d0d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:32:58.479039 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4fd2ca-f811-4f62-8b19-7818a9150d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:32:58.555157 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-9e4fd2ca-f811-4f62-8b19-7818a9150d0d None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:32:58.556254 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9e4fd2ca-f811-4f62-8b19-7818a9150d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:32:58.650480 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 594/1172] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:32:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:32:58.655110 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02a2b5b-7685-4758-8b95-9840b3cb77fb None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:32:58.656832 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e02a2b5b-7685-4758-8b95-9840b3cb77fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:32:58.668580 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02a2b5b-7685-4758-8b95-9840b3cb77fb None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:32:58.732057 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:32:58.764947 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d2d085-34fa-4a3e-953b-4d7dcf3a7c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:32:58.765549 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d2d085-34fa-4a3e-953b-4d7dcf3a7c8c admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:32:58.765826 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d2d085-34fa-4a3e-953b-4d7dcf3a7c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:32:58.766089 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d2d085-34fa-4a3e-953b-4d7dcf3a7c8c admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:32:58.766715 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d2d085-34fa-4a3e-953b-4d7dcf3a7c8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:32:58.767010 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d2d085-34fa-4a3e-953b-4d7dcf3a7c8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:32:58.768018 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: WARNING py.warnings [None req-04d2d085-34fa-4a3e-953b-4d7dcf3a7c8c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 27 09:32:58.768018 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: warnings.warn(msg) Oct 27 09:32:58.768018 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: Oct 27 09:32:58.768920 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d2d085-34fa-4a3e-953b-4d7dcf3a7c8c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:32:58.780676 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 579/1173] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:32:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:32:58.790541 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4fd2ca-f811-4f62-8b19-7818a9150d0d None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:32:58.800504 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4fd2ca-f811-4f62-8b19-7818a9150d0d None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:32:58.863964 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:32:58.898335 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b852a76-d332-4110-9287-feb4fb2894cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:32:58.898937 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b852a76-d332-4110-9287-feb4fb2894cf admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:32:58.899216 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b852a76-d332-4110-9287-feb4fb2894cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:32:58.899486 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b852a76-d332-4110-9287-feb4fb2894cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:32:58.900116 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b852a76-d332-4110-9287-feb4fb2894cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:32:58.901260 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: WARNING py.warnings [None req-5b852a76-d332-4110-9287-feb4fb2894cf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 27 09:32:58.901260 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: warnings.warn(msg) Oct 27 09:32:58.901260 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: Oct 27 09:32:58.902441 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b852a76-d332-4110-9287-feb4fb2894cf admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:32:58.956753 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 595/1174] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:32:58 2022] POST /identity/v3/projects => generated 377 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:32:58.961222 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d2d085-34fa-4a3e-953b-4d7dcf3a7c8c admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:32:58.962737 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-04d2d085-34fa-4a3e-953b-4d7dcf3a7c8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:32:58.997406 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d2d085-34fa-4a3e-953b-4d7dcf3a7c8c admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:32:59.090309 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae0eb4e-e821-46a0-879d-cfc0aa14e36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:32:59.090909 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae0eb4e-e821-46a0-879d-cfc0aa14e36c admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:32:59.091184 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae0eb4e-e821-46a0-879d-cfc0aa14e36c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:32:59.091450 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae0eb4e-e821-46a0-879d-cfc0aa14e36c admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:32:59.092061 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae0eb4e-e821-46a0-879d-cfc0aa14e36c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:32:59.093098 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: WARNING py.warnings [None req-7ae0eb4e-e821-46a0-879d-cfc0aa14e36c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 27 09:32:59.093098 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: warnings.warn(msg) Oct 27 09:32:59.093098 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: Oct 27 09:32:59.094851 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae0eb4e-e821-46a0-879d-cfc0aa14e36c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:32:59.135659 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 580/1175] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:32:58 2022] POST /identity/v3/users => generated 289 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:32:59.139684 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b852a76-d332-4110-9287-feb4fb2894cf admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:32:59.141776 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5b852a76-d332-4110-9287-feb4fb2894cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:32:59.203491 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b852a76-d332-4110-9287-feb4fb2894cf admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:32:59.284821 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:32:59.296175 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f27c7c-2b40-4bf4-a2fa-b965bff490e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:32:59.296842 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f27c7c-2b40-4bf4-a2fa-b965bff490e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:32:59.297116 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f27c7c-2b40-4bf4-a2fa-b965bff490e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:32:59.297378 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f27c7c-2b40-4bf4-a2fa-b965bff490e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:32:59.297999 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f27c7c-2b40-4bf4-a2fa-b965bff490e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:32:59.299568 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f27c7c-2b40-4bf4-a2fa-b965bff490e3 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:32:59.311544 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 596/1176] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:32:59 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 09:32:59.316171 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae0eb4e-e821-46a0-879d-cfc0aa14e36c admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:32:59.317762 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7ae0eb4e-e821-46a0-879d-cfc0aa14e36c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:32:59.329710 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae0eb4e-e821-46a0-879d-cfc0aa14e36c admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:32:59.354461 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:32:59.423085 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a683f50-9a5b-42eb-9f3a-7f672edec5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:32:59.423604 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a683f50-9a5b-42eb-9f3a-7f672edec5ac admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:32:59.423881 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a683f50-9a5b-42eb-9f3a-7f672edec5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:32:59.424147 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a683f50-9a5b-42eb-9f3a-7f672edec5ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:32:59.424808 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a683f50-9a5b-42eb-9f3a-7f672edec5ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:32:59.426395 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a683f50-9a5b-42eb-9f3a-7f672edec5ac admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:32:59.439059 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 581/1177] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:32:59 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:32:59.443155 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f27c7c-2b40-4bf4-a2fa-b965bff490e3 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:32:59.444656 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-11f27c7c-2b40-4bf4-a2fa-b965bff490e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:32:59.478444 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f27c7c-2b40-4bf4-a2fa-b965bff490e3 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:32:59.572222 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f2ac21-6034-40ab-b199-45f6d2d3779b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:32:59.572892 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2ac21-6034-40ab-b199-45f6d2d3779b admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:32:59.573169 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2ac21-6034-40ab-b199-45f6d2d3779b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:32:59.573442 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2ac21-6034-40ab-b199-45f6d2d3779b admin admin] PATH_INFO: `/v3/projects/8e6c758a2af9439ea5ac4fe85ae7fd9b/users/2f27cc4719ab4738894021fee67a6f23/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:32:59.596163 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f2ac21-6034-40ab-b199-45f6d2d3779b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e6c758a2af9439ea5ac4fe85ae7fd9b, user_id=2f27cc4719ab4738894021fee67a6f23, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:32:59.598858 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f2ac21-6034-40ab-b199-45f6d2d3779b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:32:59.623204 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 597/1178] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:32:59 2022] PUT /identity/v3/projects/8e6c758a2af9439ea5ac4fe85ae7fd9b/users/2f27cc4719ab4738894021fee67a6f23/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:32:59.634771 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652339f9-ba99-408a-a0bf-5730aae34dce None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:32:59.635062 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652339f9-ba99-408a-a0bf-5730aae34dce None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:32:59.635339 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652339f9-ba99-408a-a0bf-5730aae34dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:32:59.693639 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-652339f9-ba99-408a-a0bf-5730aae34dce None None] MFA Rules not processed for user `2f27cc4719ab4738894021fee67a6f23`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:32:59.694801 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-652339f9-ba99-408a-a0bf-5730aae34dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:32:59.771880 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 582/1179] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:32:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:32:59.784884 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f1a1f1-84ff-49aa-8896-17356768621d None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:32:59.785168 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f1a1f1-84ff-49aa-8896-17356768621d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:32:59.785436 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f1a1f1-84ff-49aa-8896-17356768621d None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:32:59.786745 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 598/1180] 10.222.0.70 () {54 vars in 963 bytes} [Thu Oct 27 09:32:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:32:59.791639 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c69cb9-dcc7-48d8-bb9c-411c5bd16dd8 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:32:59.791923 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c69cb9-dcc7-48d8-bb9c-411c5bd16dd8 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:32:59.792192 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c69cb9-dcc7-48d8-bb9c-411c5bd16dd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:32:59.824343 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:32:59.868490 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b1c69cb9-dcc7-48d8-bb9c-411c5bd16dd8 None None] MFA Rules not processed for user `5b8e13ce967449c1b12783183e953579`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:32:59.869928 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b1c69cb9-dcc7-48d8-bb9c-411c5bd16dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:32:59.944806 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 583/1181] 10.222.0.70 () {58 vars in 1060 bytes} [Thu Oct 27 09:32:59 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:32:59.954514 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f1a1f1-84ff-49aa-8896-17356768621d None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:32:59.956023 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d6f1a1f1-84ff-49aa-8896-17356768621d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:33:00.005587 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:33:00.060037 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f1a1f1-84ff-49aa-8896-17356768621d None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:33:00.136642 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ef950-24d0-4148-9296-47703422e169 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:33:00.137429 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ef950-24d0-4148-9296-47703422e169 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:33:00.137809 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ef950-24d0-4148-9296-47703422e169 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:33:00.138116 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ef950-24d0-4148-9296-47703422e169 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:33:00.139948 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5a4ef950-24d0-4148-9296-47703422e169 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:33:00.238815 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4ef950-24d0-4148-9296-47703422e169 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:33:00.240153 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4ef950-24d0-4148-9296-47703422e169 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:33:00.243643 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:33:00.258211 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 599/1182] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:32:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:33:00.303923 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5120bc3-7859-4b27-bfb1-41eba0de41e0 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:33:00.304213 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5120bc3-7859-4b27-bfb1-41eba0de41e0 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:33:00.304487 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5120bc3-7859-4b27-bfb1-41eba0de41e0 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:33:00.305759 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 584/1183] 10.222.0.70 () {52 vars in 830 bytes} [Thu Oct 27 09:33:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:33:00.309227 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ef950-24d0-4148-9296-47703422e169 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:33:00.310448 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5a4ef950-24d0-4148-9296-47703422e169 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:33:00.408679 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ef950-24d0-4148-9296-47703422e169 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:33:00.504100 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5578de-4bb3-4f51-a3e0-bb4735127853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:33:00.504675 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5578de-4bb3-4f51-a3e0-bb4735127853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:33:00.504980 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5578de-4bb3-4f51-a3e0-bb4735127853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:33:00.505248 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5578de-4bb3-4f51-a3e0-bb4735127853 admin admin] PATH_INFO: `/v3/users/2f27cc4719ab4738894021fee67a6f23` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:33:00.517821 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5578de-4bb3-4f51-a3e0-bb4735127853 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f27cc4719ab4738894021fee67a6f23)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:33:00.518865 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: WARNING py.warnings [None req-ac5578de-4bb3-4f51-a3e0-bb4735127853 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 27 09:33:00.518865 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: warnings.warn(msg) Oct 27 09:33:00.518865 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: Oct 27 09:33:00.520077 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5578de-4bb3-4f51-a3e0-bb4735127853 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:33:00.530052 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:33:00.583489 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ac5578de-4bb3-4f51-a3e0-bb4735127853 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f27cc4719ab4738894021fee67a6f23'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:33:00.599349 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ac5578de-4bb3-4f51-a3e0-bb4735127853 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f27cc4719ab4738894021fee67a6f23'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:33:00.622205 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ac5578de-4bb3-4f51-a3e0-bb4735127853 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f27cc4719ab4738894021fee67a6f23'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:33:00.630457 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ac5578de-4bb3-4f51-a3e0-bb4735127853 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f27cc4719ab4738894021fee67a6f23'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:33:00.631022 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ac5578de-4bb3-4f51-a3e0-bb4735127853 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f27cc4719ab4738894021fee67a6f23'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:33:00.636497 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 600/1184] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:33:00 2022] DELETE /identity/v3/users/2f27cc4719ab4738894021fee67a6f23 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:33:00.641541 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180f121-f83e-4944-9f2e-971c36e5281e None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:33:00.642044 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180f121-f83e-4944-9f2e-971c36e5281e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:33:00.642155 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180f121-f83e-4944-9f2e-971c36e5281e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:33:00.698822 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-9180f121-f83e-4944-9f2e-971c36e5281e None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:33:00.700004 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9180f121-f83e-4944-9f2e-971c36e5281e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:33:00.774692 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:33:00.787685 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 585/1185] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:33:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:33:00.797533 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5578de-4bb3-4f51-a3e0-bb4735127853 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:33:00.798751 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ac5578de-4bb3-4f51-a3e0-bb4735127853 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:33:00.898012 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5578de-4bb3-4f51-a3e0-bb4735127853 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:33:00.950892 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:33:00.971909 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6729831e-897e-4d6a-94f5-61e0953a3830 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:33:00.972523 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729831e-897e-4d6a-94f5-61e0953a3830 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:33:00.972826 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729831e-897e-4d6a-94f5-61e0953a3830 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:33:00.973098 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729831e-897e-4d6a-94f5-61e0953a3830 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:33:00.974672 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6729831e-897e-4d6a-94f5-61e0953a3830 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:33:01.081807 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6729831e-897e-4d6a-94f5-61e0953a3830 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:33:01.083261 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6729831e-897e-4d6a-94f5-61e0953a3830 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:33:01.103126 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 601/1186] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:33:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:33:01.451307 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9180f121-f83e-4944-9f2e-971c36e5281e None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:33:01.453110 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9180f121-f83e-4944-9f2e-971c36e5281e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:33:01.465979 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9180f121-f83e-4944-9f2e-971c36e5281e None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:33:01.564920 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba40c58d-3e4c-4b50-868c-462f05a73091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:33:01.565550 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba40c58d-3e4c-4b50-868c-462f05a73091 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:33:01.565826 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba40c58d-3e4c-4b50-868c-462f05a73091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:33:01.566092 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba40c58d-3e4c-4b50-868c-462f05a73091 admin admin] PATH_INFO: `/v3/projects/8e6c758a2af9439ea5ac4fe85ae7fd9b` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:33:01.575865 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba40c58d-3e4c-4b50-868c-462f05a73091 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e6c758a2af9439ea5ac4fe85ae7fd9b)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:33:01.576923 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: WARNING py.warnings [None req-ba40c58d-3e4c-4b50-868c-462f05a73091 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 27 09:33:01.576923 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: warnings.warn(msg) Oct 27 09:33:01.576923 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: Oct 27 09:33:01.578769 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba40c58d-3e4c-4b50-868c-462f05a73091 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:33:01.636866 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ba40c58d-3e4c-4b50-868c-462f05a73091 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e6c758a2af9439ea5ac4fe85ae7fd9b', 'request_id': 'req-ba40c58d-3e4c-4b50-868c-462f05a73091'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:33:01.650326 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ba40c58d-3e4c-4b50-868c-462f05a73091 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e6c758a2af9439ea5ac4fe85ae7fd9b', 'request_id': 'req-ba40c58d-3e4c-4b50-868c-462f05a73091'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:33:01.656475 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ba40c58d-3e4c-4b50-868c-462f05a73091 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e6c758a2af9439ea5ac4fe85ae7fd9b', 'request_id': 'req-ba40c58d-3e4c-4b50-868c-462f05a73091'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:33:01.681782 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ba40c58d-3e4c-4b50-868c-462f05a73091 admin admin] The token cache is being invalidate because project 8e6c758a2af9439ea5ac4fe85ae7fd9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:33:01.682073 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ba40c58d-3e4c-4b50-868c-462f05a73091 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:33:01.682617 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ba40c58d-3e4c-4b50-868c-462f05a73091 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:33:01.683934 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 586/1187] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:33:01 2022] DELETE /identity/v3/projects/8e6c758a2af9439ea5ac4fe85ae7fd9b => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:33:11.100803 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=62 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:33:11.679770 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:34:30.801031 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519c9fb-5381-4e2a-b00a-31337d3735e5 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:34:30.801664 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519c9fb-5381-4e2a-b00a-31337d3735e5 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:34:30.801664 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519c9fb-5381-4e2a-b00a-31337d3735e5 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:34:30.802949 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 602/1188] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 27 09:34:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:34:30.807767 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2b4b9-d459-4923-93b8-2e6f3b9c6f88 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:34:30.808204 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2b4b9-d459-4923-93b8-2e6f3b9c6f88 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:34:30.808303 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2b4b9-d459-4923-93b8-2e6f3b9c6f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:34:30.888675 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-8ff2b4b9-d459-4923-93b8-2e6f3b9c6f88 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:34:30.889885 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8ff2b4b9-d459-4923-93b8-2e6f3b9c6f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:34:30.993997 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 587/1189] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 27 09:34:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:34:31.148450 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0519c9fb-5381-4e2a-b00a-31337d3735e5 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:34:31.150599 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0519c9fb-5381-4e2a-b00a-31337d3735e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:34:31.267236 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0519c9fb-5381-4e2a-b00a-31337d3735e5 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:34:31.272756 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c90889a-4393-49f0-ad42-3ed442d87417 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe4a6e4b6184826b30ac4fc13ad9b1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:34:31.273361 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c90889a-4393-49f0-ad42-3ed442d87417 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:34:31.273635 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c90889a-4393-49f0-ad42-3ed442d87417 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:34:31.273902 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c90889a-4393-49f0-ad42-3ed442d87417 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:34:31.282287 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c90889a-4393-49f0-ad42-3ed442d87417 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:34:31.283879 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c90889a-4393-49f0-ad42-3ed442d87417 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:34:31.293183 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 603/1190] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:34:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:34:31.892808 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc74445-1663-48b1-8eaf-7fa9a779dd5a None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:34:31.893376 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc74445-1663-48b1-8eaf-7fa9a779dd5a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:34:31.893376 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc74445-1663-48b1-8eaf-7fa9a779dd5a None None] PATH_INFO: `/v3` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:34:31.894620 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 588/1191] 10.222.0.72 () {54 vars in 925 bytes} [Thu Oct 27 09:34:31 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:34:31.901048 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf73c3-0050-4c34-87f9-9f3ae4c4cda0 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:34:31.901480 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf73c3-0050-4c34-87f9-9f3ae4c4cda0 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:34:31.901612 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf73c3-0050-4c34-87f9-9f3ae4c4cda0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:34:31.960493 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-2abf73c3-0050-4c34-87f9-9f3ae4c4cda0 None None] MFA Rules not processed for user `2316b30a6e7b4175bd7b90583584eab5`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:34:31.961712 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2abf73c3-0050-4c34-87f9-9f3ae4c4cda0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:34:32.029624 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:34:32.035136 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 604/1192] 10.222.0.72 () {58 vars in 1015 bytes} [Thu Oct 27 09:34:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:34:32.367559 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c44c2-05ea-458b-94fd-e9a5fd521497 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:34:32.367839 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c44c2-05ea-458b-94fd-e9a5fd521497 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:34:32.368111 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c44c2-05ea-458b-94fd-e9a5fd521497 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:34:32.369477 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 589/1193] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 27 09:34:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:34:32.374815 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f439a66-61ee-40ce-ac88-dc36137207df None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:34:32.375106 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f439a66-61ee-40ce-ac88-dc36137207df None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:34:32.375416 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f439a66-61ee-40ce-ac88-dc36137207df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:34:32.395294 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-9f439a66-61ee-40ce-ac88-dc36137207df None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:34:32.396498 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9f439a66-61ee-40ce-ac88-dc36137207df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:34:32.404305 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 605/1194] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 27 09:34:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:34:32.553955 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c44c2-05ea-458b-94fd-e9a5fd521497 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:34:32.568536 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c44c2-05ea-458b-94fd-e9a5fd521497 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:34:32.574017 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b2ee88-2dd8-4c58-b56d-d567840e6d2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe4a6e4b6184826b30ac4fc13ad9b1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:34:32.574737 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2ee88-2dd8-4c58-b56d-d567840e6d2c service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:34:32.575539 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2ee88-2dd8-4c58-b56d-d567840e6d2c service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:34:32.575811 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2ee88-2dd8-4c58-b56d-d567840e6d2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:34:32.584095 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b2ee88-2dd8-4c58-b56d-d567840e6d2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:34:32.585726 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b2ee88-2dd8-4c58-b56d-d567840e6d2c service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:34:32.594975 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 590/1195] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:34:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:34:35.673671 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a6c93-e75e-4bea-9791-cafca43944cd None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:34:35.674135 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a6c93-e75e-4bea-9791-cafca43944cd None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:34:35.674244 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a6c93-e75e-4bea-9791-cafca43944cd None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:34:35.675601 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 606/1196] 10.222.0.70 () {54 vars in 961 bytes} [Thu Oct 27 09:34:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:34:35.680808 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72178950-a823-4000-9b49-38643534e61b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:34:35.681221 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72178950-a823-4000-9b49-38643534e61b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:34:35.681357 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72178950-a823-4000-9b49-38643534e61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:34:35.701384 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-72178950-a823-4000-9b49-38643534e61b None None] MFA Rules not processed for user `2316b30a6e7b4175bd7b90583584eab5`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:34:35.702588 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-72178950-a823-4000-9b49-38643534e61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:34:35.711238 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 591/1197] 10.222.0.70 () {58 vars in 1058 bytes} [Thu Oct 27 09:34:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:34:35.718784 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a6c93-e75e-4bea-9791-cafca43944cd None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:34:35.732303 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a6c93-e75e-4bea-9791-cafca43944cd None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:34:35.737238 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d869dbbe-2321-4898-9438-d11eb59a3724 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2316b30a6e7b4175bd7b90583584eab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:34:35.737841 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d869dbbe-2321-4898-9438-d11eb59a3724 service placement] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:34:35.738116 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d869dbbe-2321-4898-9438-d11eb59a3724 service placement] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:34:35.738384 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d869dbbe-2321-4898-9438-d11eb59a3724 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:34:35.746362 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d869dbbe-2321-4898-9438-d11eb59a3724 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:34:35.748696 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d869dbbe-2321-4898-9438-d11eb59a3724 service placement] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:34:35.757976 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 607/1198] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:34:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:34:44.044126 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd78e6-3b50-45be-9414-3adcabf35386 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:34:44.044746 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd78e6-3b50-45be-9414-3adcabf35386 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:34:44.044746 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd78e6-3b50-45be-9414-3adcabf35386 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:34:44.046556 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 592/1199] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 27 09:34:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:34:44.051263 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b3f54-25cb-4c66-9aa0-2df38bbdf838 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:34:44.051662 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b3f54-25cb-4c66-9aa0-2df38bbdf838 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:34:44.051817 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b3f54-25cb-4c66-9aa0-2df38bbdf838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:34:44.071387 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-055b3f54-25cb-4c66-9aa0-2df38bbdf838 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:34:44.072605 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-055b3f54-25cb-4c66-9aa0-2df38bbdf838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:34:44.080295 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 608/1200] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 27 09:34:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:34:44.235837 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dd78e6-3b50-45be-9414-3adcabf35386 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:34:44.250227 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dd78e6-3b50-45be-9414-3adcabf35386 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:34:44.255842 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3bb6d7-dcb2-484f-9c3d-9c336844df63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe4a6e4b6184826b30ac4fc13ad9b1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:34:44.256432 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3bb6d7-dcb2-484f-9c3d-9c336844df63 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:34:44.256738 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3bb6d7-dcb2-484f-9c3d-9c336844df63 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:34:44.257010 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3bb6d7-dcb2-484f-9c3d-9c336844df63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:34:44.264969 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3bb6d7-dcb2-484f-9c3d-9c336844df63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:34:44.266546 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3bb6d7-dcb2-484f-9c3d-9c336844df63 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:34:44.276077 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 593/1201] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:34:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:34:45.335152 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7537b-b348-40e8-b8bc-5268e83032fa None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:34:45.335628 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7537b-b348-40e8-b8bc-5268e83032fa None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:34:45.335746 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7537b-b348-40e8-b8bc-5268e83032fa None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:34:45.337112 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 609/1202] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 27 09:34:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:34:45.341791 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f028a7-3754-4dc5-b508-24e6bfe2fe4f None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:34:45.342249 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f028a7-3754-4dc5-b508-24e6bfe2fe4f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:34:45.342347 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f028a7-3754-4dc5-b508-24e6bfe2fe4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:34:45.361709 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-86f028a7-3754-4dc5-b508-24e6bfe2fe4f None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:34:45.362881 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-86f028a7-3754-4dc5-b508-24e6bfe2fe4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:34:45.370524 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 594/1203] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 27 09:34:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:34:45.519906 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a7537b-b348-40e8-b8bc-5268e83032fa None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:34:45.534081 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a7537b-b348-40e8-b8bc-5268e83032fa None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:34:45.539519 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf5522d-4282-4957-921d-7e997cc61210 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe4a6e4b6184826b30ac4fc13ad9b1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:34:45.540052 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf5522d-4282-4957-921d-7e997cc61210 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:34:45.540330 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf5522d-4282-4957-921d-7e997cc61210 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:34:45.540628 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf5522d-4282-4957-921d-7e997cc61210 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:34:45.549095 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf5522d-4282-4957-921d-7e997cc61210 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:34:45.550652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf5522d-4282-4957-921d-7e997cc61210 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:34:45.559860 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 610/1204] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:34:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:34:49.713499 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbfff8-7aaf-4cd5-842b-967c3f739536 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:34:49.714080 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbfff8-7aaf-4cd5-842b-967c3f739536 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:34:49.714080 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbfff8-7aaf-4cd5-842b-967c3f739536 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:34:49.715426 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 595/1205] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 27 09:34:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:34:49.721262 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f3277-6719-48d4-9ac1-68433def9f68 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:34:49.721665 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f3277-6719-48d4-9ac1-68433def9f68 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:34:49.721814 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f3277-6719-48d4-9ac1-68433def9f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:34:49.741602 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-645f3277-6719-48d4-9ac1-68433def9f68 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:34:49.742791 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-645f3277-6719-48d4-9ac1-68433def9f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:34:49.750802 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 611/1206] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 27 09:34:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:34:50.042589 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dbfff8-7aaf-4cd5-842b-967c3f739536 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:34:50.044266 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-15dbfff8-7aaf-4cd5-842b-967c3f739536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:34:50.077058 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:34:50.124200 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dbfff8-7aaf-4cd5-842b-967c3f739536 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:34:50.127767 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0159ea69-1226-4426-99a1-60b87079ecc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:34:50.128362 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0159ea69-1226-4426-99a1-60b87079ecc0 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:34:50.128672 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0159ea69-1226-4426-99a1-60b87079ecc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:34:50.128945 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0159ea69-1226-4426-99a1-60b87079ecc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:34:50.137676 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0159ea69-1226-4426-99a1-60b87079ecc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:34:50.139625 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0159ea69-1226-4426-99a1-60b87079ecc0 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:34:50.150923 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 596/1207] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:34:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:34:52.293594 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8bfa3-65db-4906-9510-fda9d4106e64 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:34:52.294100 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8bfa3-65db-4906-9510-fda9d4106e64 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:34:52.294208 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8bfa3-65db-4906-9510-fda9d4106e64 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:34:52.295998 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 612/1208] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 27 09:34:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:34:52.301286 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b519a6-978d-4036-8e25-05897197153f None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:34:52.301685 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b519a6-978d-4036-8e25-05897197153f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:34:52.301823 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b519a6-978d-4036-8e25-05897197153f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:34:52.321317 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-12b519a6-978d-4036-8e25-05897197153f None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:34:52.322485 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-12b519a6-978d-4036-8e25-05897197153f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:34:52.330171 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 597/1209] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 27 09:34:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:34:52.612828 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c8bfa3-65db-4906-9510-fda9d4106e64 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:34:52.623395 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c8bfa3-65db-4906-9510-fda9d4106e64 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:34:52.626866 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e47a00-98d3-4984-872c-8f905b8570ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:34:52.627454 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e47a00-98d3-4984-872c-8f905b8570ca service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:34:52.627734 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e47a00-98d3-4984-872c-8f905b8570ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:34:52.628003 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e47a00-98d3-4984-872c-8f905b8570ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:34:52.636606 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e47a00-98d3-4984-872c-8f905b8570ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:34:52.638043 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e47a00-98d3-4984-872c-8f905b8570ca service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:34:52.648074 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 613/1210] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:34:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:34:56.350677 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886ce83-f26d-40fb-ba47-1bbf579f2b03 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:34:56.351234 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886ce83-f26d-40fb-ba47-1bbf579f2b03 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:34:56.351234 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886ce83-f26d-40fb-ba47-1bbf579f2b03 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:34:56.352629 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 598/1211] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 27 09:34:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:34:56.358082 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6909a54-cf40-43bc-a86a-3ce897d4efe7 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:34:56.359100 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6909a54-cf40-43bc-a86a-3ce897d4efe7 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:34:56.359257 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6909a54-cf40-43bc-a86a-3ce897d4efe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:34:56.378695 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-b6909a54-cf40-43bc-a86a-3ce897d4efe7 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:34:56.379895 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b6909a54-cf40-43bc-a86a-3ce897d4efe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:34:56.387818 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 614/1212] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 27 09:34:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:34:56.696951 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d886ce83-f26d-40fb-ba47-1bbf579f2b03 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:34:56.698587 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d886ce83-f26d-40fb-ba47-1bbf579f2b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:34:56.778906 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d886ce83-f26d-40fb-ba47-1bbf579f2b03 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:34:56.783137 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b141085-ec40-4691-b673-20e46dac92e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:34:56.783738 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b141085-ec40-4691-b673-20e46dac92e9 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:34:56.784018 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b141085-ec40-4691-b673-20e46dac92e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:34:56.784284 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b141085-ec40-4691-b673-20e46dac92e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:34:56.793120 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b141085-ec40-4691-b673-20e46dac92e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:34:56.794551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b141085-ec40-4691-b673-20e46dac92e9 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:34:56.803735 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 599/1213] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:34:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:34:58.946916 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c866c9-1588-44e8-90fc-6723a422975d None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:34:58.947450 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c866c9-1588-44e8-90fc-6723a422975d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:34:58.947540 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c866c9-1588-44e8-90fc-6723a422975d None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:34:58.948897 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 615/1214] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 27 09:34:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:34:58.954837 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6747337-2d7f-4e82-99df-0a992a49896c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:34:58.955261 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6747337-2d7f-4e82-99df-0a992a49896c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:34:58.955366 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6747337-2d7f-4e82-99df-0a992a49896c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:34:58.975619 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-d6747337-2d7f-4e82-99df-0a992a49896c None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:34:58.976851 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d6747337-2d7f-4e82-99df-0a992a49896c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:34:58.984832 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 600/1215] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 27 09:34:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:34:59.281912 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c866c9-1588-44e8-90fc-6723a422975d None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:34:59.283599 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-60c866c9-1588-44e8-90fc-6723a422975d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:34:59.296381 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c866c9-1588-44e8-90fc-6723a422975d None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:34:59.299888 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35690a4f-a946-419c-a71b-464ec7669bdb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:34:59.300474 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35690a4f-a946-419c-a71b-464ec7669bdb service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:34:59.300778 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35690a4f-a946-419c-a71b-464ec7669bdb service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:34:59.301047 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35690a4f-a946-419c-a71b-464ec7669bdb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:34:59.309671 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35690a4f-a946-419c-a71b-464ec7669bdb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:34:59.311081 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35690a4f-a946-419c-a71b-464ec7669bdb service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:34:59.320735 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 616/1216] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:34:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:35:08.970748 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=63 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:35:09.316627 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=37 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:24.073501 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e1f30-7491-48c6-9071-d5802dde77db None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:24.075272 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e1f30-7491-48c6-9071-d5802dde77db None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:24.075272 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e1f30-7491-48c6-9071-d5802dde77db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:24.098420 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c94de13-4aa8-45c2-bea2-aaa0c83c4af9 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:24.099020 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c94de13-4aa8-45c2-bea2-aaa0c83c4af9 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:24.099020 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c94de13-4aa8-45c2-bea2-aaa0c83c4af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:24.103403 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-5d7e1f30-7491-48c6-9071-d5802dde77db None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:24.104912 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5d7e1f30-7491-48c6-9071-d5802dde77db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:24.119423 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 601/1217] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:24.129216 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f3a6d-c07f-4992-bd0c-bbabd1391dc8 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:24.129885 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f3a6d-c07f-4992-bd0c-bbabd1391dc8 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:24.132852 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f3a6d-c07f-4992-bd0c-bbabd1391dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:24.135484 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-3c94de13-4aa8-45c2-bea2-aaa0c83c4af9 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:24.137035 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3c94de13-4aa8-45c2-bea2-aaa0c83c4af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:24.162844 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-6f1f3a6d-c07f-4992-bd0c-bbabd1391dc8 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:24.165881 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6f1f3a6d-c07f-4992-bd0c-bbabd1391dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:24.169044 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 617/1218] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:36:24.180102 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d96b1-5881-486c-bf59-31f8141b1a8b None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:24.180473 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d96b1-5881-486c-bf59-31f8141b1a8b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:24.180860 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d96b1-5881-486c-bf59-31f8141b1a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:24.185332 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 602/1219] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:24.192477 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f3a6d-c07f-4992-bd0c-bbabd1391dc8 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:24.209490 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f3a6d-c07f-4992-bd0c-bbabd1391dc8 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:24.224764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae7813a-9ed0-4f11-a8a0-535b7276e2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:24.225399 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae7813a-9ed0-4f11-a8a0-535b7276e2e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:24.225679 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae7813a-9ed0-4f11-a8a0-535b7276e2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:24.225950 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae7813a-9ed0-4f11-a8a0-535b7276e2e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:24.226603 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae7813a-9ed0-4f11-a8a0-535b7276e2e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:36:24.227043 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae7813a-9ed0-4f11-a8a0-535b7276e2e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:24.228698 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae7813a-9ed0-4f11-a8a0-535b7276e2e6 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:24.229654 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-601d96b1-5881-486c-bf59-31f8141b1a8b None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:24.230884 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-601d96b1-5881-486c-bf59-31f8141b1a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:24.249529 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 618/1220] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:36:24.254380 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601d96b1-5881-486c-bf59-31f8141b1a8b None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:24.259349 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 603/1221] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:36:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:36:24.268958 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601d96b1-5881-486c-bf59-31f8141b1a8b None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:24.273933 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae7813a-9ed0-4f11-a8a0-535b7276e2e6 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:24.285318 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373f6e23-f58b-4c47-99a4-e2e1060795ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:24.285962 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373f6e23-f58b-4c47-99a4-e2e1060795ff admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:24.286245 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373f6e23-f58b-4c47-99a4-e2e1060795ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:24.286515 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373f6e23-f58b-4c47-99a4-e2e1060795ff admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:24.287166 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373f6e23-f58b-4c47-99a4-e2e1060795ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:36:24.287466 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373f6e23-f58b-4c47-99a4-e2e1060795ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:24.288485 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: WARNING py.warnings [None req-373f6e23-f58b-4c47-99a4-e2e1060795ff admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 27 09:36:24.288485 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: warnings.warn(msg) Oct 27 09:36:24.288485 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: Oct 27 09:36:24.289813 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373f6e23-f58b-4c47-99a4-e2e1060795ff admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:24.302036 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae7813a-9ed0-4f11-a8a0-535b7276e2e6 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:24.308043 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 619/1222] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:36:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:36:24.309585 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00143393-154b-4f9d-a436-db5bb1062aa9 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:24.309878 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00143393-154b-4f9d-a436-db5bb1062aa9 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:24.310170 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00143393-154b-4f9d-a436-db5bb1062aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:24.320351 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec243b9-8c87-4f6e-a95b-41884c1f3d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:24.321260 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec243b9-8c87-4f6e-a95b-41884c1f3d56 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:24.321726 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec243b9-8c87-4f6e-a95b-41884c1f3d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:24.322209 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec243b9-8c87-4f6e-a95b-41884c1f3d56 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:24.323047 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec243b9-8c87-4f6e-a95b-41884c1f3d56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:24.324332 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: WARNING py.warnings [None req-4ec243b9-8c87-4f6e-a95b-41884c1f3d56 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 27 09:36:24.324332 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: warnings.warn(msg) Oct 27 09:36:24.324332 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: Oct 27 09:36:24.325912 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec243b9-8c87-4f6e-a95b-41884c1f3d56 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:24.361373 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-00143393-154b-4f9d-a436-db5bb1062aa9 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:24.361373 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-00143393-154b-4f9d-a436-db5bb1062aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:24.373683 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 620/1223] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:24.376540 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00143393-154b-4f9d-a436-db5bb1062aa9 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:24.393566 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00143393-154b-4f9d-a436-db5bb1062aa9 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:24.406744 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c1d56e-00a4-4ba3-b6c9-8e37fd791185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:24.407408 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1d56e-00a4-4ba3-b6c9-8e37fd791185 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:24.407687 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1d56e-00a4-4ba3-b6c9-8e37fd791185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:24.407957 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1d56e-00a4-4ba3-b6c9-8e37fd791185 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:24.408620 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c1d56e-00a4-4ba3-b6c9-8e37fd791185 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:24.410519 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c1d56e-00a4-4ba3-b6c9-8e37fd791185 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:24.418947 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 604/1224] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:36:24 2022] POST /identity/v3/projects => generated 371 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:36:24.421967 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212b1da-a193-4564-a209-5194f4b7a6a1 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:24.422552 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212b1da-a193-4564-a209-5194f4b7a6a1 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:24.423101 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212b1da-a193-4564-a209-5194f4b7a6a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:24.455468 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-c212b1da-a193-4564-a209-5194f4b7a6a1 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:24.462315 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c212b1da-a193-4564-a209-5194f4b7a6a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:24.466801 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 621/1225] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:36:24 2022] POST /identity/v3/projects => generated 381 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:36:24.468884 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2f87b-3c58-4bd4-aa77-80a89bea8483 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:24.469425 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2f87b-3c58-4bd4-aa77-80a89bea8483 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:24.470383 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2f87b-3c58-4bd4-aa77-80a89bea8483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:24.495739 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-eef2f87b-3c58-4bd4-aa77-80a89bea8483 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:24.497009 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-eef2f87b-3c58-4bd4-aa77-80a89bea8483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:24.532599 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 605/1226] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:24.533794 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212b1da-a193-4564-a209-5194f4b7a6a1 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:24.550568 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212b1da-a193-4564-a209-5194f4b7a6a1 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:24.557411 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 622/1227] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:24.558686 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbec222-6133-47b3-b124-1e699c32b239 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:24.558995 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbec222-6133-47b3-b124-1e699c32b239 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:24.559268 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbec222-6133-47b3-b124-1e699c32b239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:24.579762 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-2fbec222-6133-47b3-b124-1e699c32b239 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:24.580993 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2fbec222-6133-47b3-b124-1e699c32b239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:24.589246 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 623/1228] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:24.590414 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca73d5-abbe-417c-ade9-f4fa52774352 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:24.590698 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca73d5-abbe-417c-ade9-f4fa52774352 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:24.590969 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca73d5-abbe-417c-ade9-f4fa52774352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:24.609773 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-c7ca73d5-abbe-417c-ade9-f4fa52774352 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:24.611510 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c7ca73d5-abbe-417c-ade9-f4fa52774352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:24.615704 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa16b097-a929-4e3b-819d-d87ab473e40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:24.616348 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa16b097-a929-4e3b-819d-d87ab473e40e admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:24.616660 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa16b097-a929-4e3b-819d-d87ab473e40e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:24.616935 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa16b097-a929-4e3b-819d-d87ab473e40e admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:24.617593 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa16b097-a929-4e3b-819d-d87ab473e40e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:24.619384 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 624/1229] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:24.619592 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa16b097-a929-4e3b-819d-d87ab473e40e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:24.620185 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ca73d5-abbe-417c-ade9-f4fa52774352 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:24.634818 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ca73d5-abbe-417c-ade9-f4fa52774352 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:24.640623 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a005a4b-8092-49ca-83f5-3ae67f6842f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:24.641227 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a005a4b-8092-49ca-83f5-3ae67f6842f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:24.641529 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a005a4b-8092-49ca-83f5-3ae67f6842f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:24.641796 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a005a4b-8092-49ca-83f5-3ae67f6842f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:24.642447 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a005a4b-8092-49ca-83f5-3ae67f6842f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:24.642956 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 606/1230] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:36:24 2022] POST /identity/v3/users => generated 286 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:36:24.643608 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: WARNING py.warnings [None req-3a005a4b-8092-49ca-83f5-3ae67f6842f4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 27 09:36:24.643608 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: warnings.warn(msg) Oct 27 09:36:24.643608 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: Oct 27 09:36:24.644387 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb9768a-d008-4eb2-b283-e05d78eedb6b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:24.644841 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a005a4b-8092-49ca-83f5-3ae67f6842f4 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:24.645459 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb9768a-d008-4eb2-b283-e05d78eedb6b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:24.645881 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb9768a-d008-4eb2-b283-e05d78eedb6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:24.665830 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-7fb9768a-d008-4eb2-b283-e05d78eedb6b None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:24.667136 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7fb9768a-d008-4eb2-b283-e05d78eedb6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:24.671303 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 625/1231] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:36:24 2022] POST /identity/v3/users => generated 291 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:36:24.671303 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ccc57-ebe3-47e8-b834-88ad1f39b42f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:24.671303 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ccc57-ebe3-47e8-b834-88ad1f39b42f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:24.671303 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ccc57-ebe3-47e8-b834-88ad1f39b42f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:24.676055 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 607/1232] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:24.677267 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf7a891-5b8f-4413-b5c7-94196bfb96c8 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:24.677546 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf7a891-5b8f-4413-b5c7-94196bfb96c8 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:24.677820 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf7a891-5b8f-4413-b5c7-94196bfb96c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:24.691384 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-ff4ccc57-ebe3-47e8-b834-88ad1f39b42f None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:24.692663 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ff4ccc57-ebe3-47e8-b834-88ad1f39b42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:24.698528 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-8cf7a891-5b8f-4413-b5c7-94196bfb96c8 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:24.699703 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8cf7a891-5b8f-4413-b5c7-94196bfb96c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:24.701218 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 626/1233] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:24.701979 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ccc57-ebe3-47e8-b834-88ad1f39b42f None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:24.711634 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ccc57-ebe3-47e8-b834-88ad1f39b42f None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:24.716181 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 608/1234] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:24.716337 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880d55a2-0242-4108-99b7-1fb630cd5b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:24.716818 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d55a2-0242-4108-99b7-1fb630cd5b8f admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:24.717104 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d55a2-0242-4108-99b7-1fb630cd5b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:24.717382 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d55a2-0242-4108-99b7-1fb630cd5b8f admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:24.717565 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f1ae4-6f99-4c88-85db-cd1cec2607aa None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:24.717849 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f1ae4-6f99-4c88-85db-cd1cec2607aa None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:24.718042 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880d55a2-0242-4108-99b7-1fb630cd5b8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:36:24.718152 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f1ae4-6f99-4c88-85db-cd1cec2607aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:24.718316 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880d55a2-0242-4108-99b7-1fb630cd5b8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:24.719949 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880d55a2-0242-4108-99b7-1fb630cd5b8f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:24.739717 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 627/1235] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:36:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:36:24.739717 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c59c3-c697-4571-b975-a54708a827b2 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:24.739717 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c59c3-c697-4571-b975-a54708a827b2 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:24.739717 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c59c3-c697-4571-b975-a54708a827b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:24.740155 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-f27f1ae4-6f99-4c88-85db-cd1cec2607aa None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:24.740155 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f27f1ae4-6f99-4c88-85db-cd1cec2607aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:24.751535 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 609/1236] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:24.751535 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27f1ae4-6f99-4c88-85db-cd1cec2607aa None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:24.754891 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-c94c59c3-c697-4571-b975-a54708a827b2 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:24.756086 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c94c59c3-c697-4571-b975-a54708a827b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:24.758665 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27f1ae4-6f99-4c88-85db-cd1cec2607aa None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:24.763327 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25cbf1d-1ec3-4d8e-a05e-fa43ee16a22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:24.763911 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25cbf1d-1ec3-4d8e-a05e-fa43ee16a22f admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:24.767639 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 628/1237] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:24.767770 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25cbf1d-1ec3-4d8e-a05e-fa43ee16a22f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:24.767770 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25cbf1d-1ec3-4d8e-a05e-fa43ee16a22f admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:24.767947 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c59c3-c697-4571-b975-a54708a827b2 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:24.768048 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25cbf1d-1ec3-4d8e-a05e-fa43ee16a22f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:24.768048 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25cbf1d-1ec3-4d8e-a05e-fa43ee16a22f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:24.773842 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c59c3-c697-4571-b975-a54708a827b2 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:24.777582 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 610/1238] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:36:24.778440 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a61fe0c-b6c7-4bae-842f-053d9ea8ad7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:24.778622 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389535d5-2328-486e-94f7-29b826c2cbae None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:24.778906 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389535d5-2328-486e-94f7-29b826c2cbae None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:24.779025 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a61fe0c-b6c7-4bae-842f-053d9ea8ad7f admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:24.779179 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389535d5-2328-486e-94f7-29b826c2cbae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:24.779306 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a61fe0c-b6c7-4bae-842f-053d9ea8ad7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:24.779582 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a61fe0c-b6c7-4bae-842f-053d9ea8ad7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:24.780210 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a61fe0c-b6c7-4bae-842f-053d9ea8ad7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:24.781833 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a61fe0c-b6c7-4bae-842f-053d9ea8ad7f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:24.786571 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:24.794997 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 629/1239] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:36:24.794997 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d50425d-4829-48b1-a7dd-b766e0413c62 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:24.794997 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d50425d-4829-48b1-a7dd-b766e0413c62 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:24.794997 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d50425d-4829-48b1-a7dd-b766e0413c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:24.800564 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-389535d5-2328-486e-94f7-29b826c2cbae None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:24.801764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-389535d5-2328-486e-94f7-29b826c2cbae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:24.809623 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 611/1240] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:24.813068 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389535d5-2328-486e-94f7-29b826c2cbae None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:24.814664 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-8d50425d-4829-48b1-a7dd-b766e0413c62 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:24.815851 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8d50425d-4829-48b1-a7dd-b766e0413c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:24.820026 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389535d5-2328-486e-94f7-29b826c2cbae None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:24.823850 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 630/1241] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:24.824455 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d50425d-4829-48b1-a7dd-b766e0413c62 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:24.824626 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f85d279-e435-4d6f-81a5-fcf3c7b0891a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:24.825210 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85d279-e435-4d6f-81a5-fcf3c7b0891a admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:24.825487 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85d279-e435-4d6f-81a5-fcf3c7b0891a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:24.825760 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85d279-e435-4d6f-81a5-fcf3c7b0891a admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:24.826393 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f85d279-e435-4d6f-81a5-fcf3c7b0891a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:36:24.826697 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f85d279-e435-4d6f-81a5-fcf3c7b0891a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:24.828293 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f85d279-e435-4d6f-81a5-fcf3c7b0891a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:24.834007 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d50425d-4829-48b1-a7dd-b766e0413c62 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:24.838536 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e49988f-0983-43e1-b227-31fc21d10ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:24.839116 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e49988f-0983-43e1-b227-31fc21d10ea8 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:24.839397 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e49988f-0983-43e1-b227-31fc21d10ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:24.839668 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e49988f-0983-43e1-b227-31fc21d10ea8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:24.840566 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e49988f-0983-43e1-b227-31fc21d10ea8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:24.840674 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 612/1242] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:36:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:36:24.841199 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f85d279-e435-4d6f-81a5-fcf3c7b0891a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:24.842356 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e49988f-0983-43e1-b227-31fc21d10ea8 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:24.853079 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f85d279-e435-4d6f-81a5-fcf3c7b0891a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:24.857638 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f047f5-542d-4123-b909-4e1a2c05d831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:24.858233 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f047f5-542d-4123-b909-4e1a2c05d831 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:24.858513 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f047f5-542d-4123-b909-4e1a2c05d831 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:24.858782 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f047f5-542d-4123-b909-4e1a2c05d831 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:24.859415 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f047f5-542d-4123-b909-4e1a2c05d831 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:36:24.859745 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f047f5-542d-4123-b909-4e1a2c05d831 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:24.861877 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f047f5-542d-4123-b909-4e1a2c05d831 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:24.871244 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 631/1243] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:36:24 2022] POST /identity/v3/projects => generated 357 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:36:24.872066 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e49988f-0983-43e1-b227-31fc21d10ea8 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:24.875314 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 613/1244] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:36:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:36:24.876188 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f047f5-542d-4123-b909-4e1a2c05d831 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:24.881708 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:24.886419 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f047f5-542d-4123-b909-4e1a2c05d831 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:24.936294 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e49988f-0983-43e1-b227-31fc21d10ea8 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:24.941086 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6fd23-cee9-4761-85e2-a16e01a5d01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:24.941697 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6fd23-cee9-4761-85e2-a16e01a5d01f admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:24.941826 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bebbab0-a1d2-437e-8ead-fd00b647c0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:24.941975 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6fd23-cee9-4761-85e2-a16e01a5d01f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:24.942248 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6fd23-cee9-4761-85e2-a16e01a5d01f admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:24.942355 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bebbab0-a1d2-437e-8ead-fd00b647c0e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:24.942606 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bebbab0-a1d2-437e-8ead-fd00b647c0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:24.942876 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bebbab0-a1d2-437e-8ead-fd00b647c0e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:24.942973 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a6fd23-cee9-4761-85e2-a16e01a5d01f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:36:24.943174 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a6fd23-cee9-4761-85e2-a16e01a5d01f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:24.943504 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bebbab0-a1d2-437e-8ead-fd00b647c0e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:24.944811 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a6fd23-cee9-4761-85e2-a16e01a5d01f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:24.945260 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bebbab0-a1d2-437e-8ead-fd00b647c0e0 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:24.955389 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 614/1245] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:36:24.956689 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bebbab0-a1d2-437e-8ead-fd00b647c0e0 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:24.956820 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 632/1246] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:36:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:36:24.957359 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6fd23-cee9-4761-85e2-a16e01a5d01f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:24.970365 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6fd23-cee9-4761-85e2-a16e01a5d01f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:24.971632 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bebbab0-a1d2-437e-8ead-fd00b647c0e0 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:24.975036 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45339fe1-e8b3-4f06-aca5-65c2ddce95b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:24.975654 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45339fe1-e8b3-4f06-aca5-65c2ddce95b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:24.975908 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45339fe1-e8b3-4f06-aca5-65c2ddce95b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:24.976188 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45339fe1-e8b3-4f06-aca5-65c2ddce95b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:24.976717 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec1eaf-5c43-43a2-9b16-b0b2ff73610d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:24.976859 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45339fe1-e8b3-4f06-aca5-65c2ddce95b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:36:24.977108 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45339fe1-e8b3-4f06-aca5-65c2ddce95b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:24.977313 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec1eaf-5c43-43a2-9b16-b0b2ff73610d admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:24.977617 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec1eaf-5c43-43a2-9b16-b0b2ff73610d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:24.978047 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec1eaf-5c43-43a2-9b16-b0b2ff73610d admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:24.978703 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45339fe1-e8b3-4f06-aca5-65c2ddce95b2 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:24.979078 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec1eaf-5c43-43a2-9b16-b0b2ff73610d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:24.981333 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec1eaf-5c43-43a2-9b16-b0b2ff73610d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:24.992052 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 615/1247] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:36:24.992197 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 633/1248] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:36:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:36:24.992754 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45339fe1-e8b3-4f06-aca5-65c2ddce95b2 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:24.992886 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec1eaf-5c43-43a2-9b16-b0b2ff73610d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:25.003907 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec1eaf-5c43-43a2-9b16-b0b2ff73610d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:25.004613 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45339fe1-e8b3-4f06-aca5-65c2ddce95b2 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:25.008479 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb825b-43fc-4123-84f5-933454fddcb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:25.009065 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb825b-43fc-4123-84f5-933454fddcb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:25.009264 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e071bbe5-146c-429d-abb2-85a03a40a1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:25.009395 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb825b-43fc-4123-84f5-933454fddcb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:25.009599 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb825b-43fc-4123-84f5-933454fddcb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:25.009877 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e071bbe5-146c-429d-abb2-85a03a40a1c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:25.010156 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e071bbe5-146c-429d-abb2-85a03a40a1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:25.010272 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebb825b-43fc-4123-84f5-933454fddcb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:36:25.010429 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e071bbe5-146c-429d-abb2-85a03a40a1c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:25.010622 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebb825b-43fc-4123-84f5-933454fddcb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:25.011054 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e071bbe5-146c-429d-abb2-85a03a40a1c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:25.012228 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebb825b-43fc-4123-84f5-933454fddcb7 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:25.012974 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e071bbe5-146c-429d-abb2-85a03a40a1c9 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:25.024494 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 616/1249] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:36:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:36:25.027500 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb825b-43fc-4123-84f5-933454fddcb7 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:25.037891 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb825b-43fc-4123-84f5-933454fddcb7 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:25.040285 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 634/1250] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:36:24 2022] POST /identity/v3/projects => generated 389 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:36:25.041136 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e071bbe5-146c-429d-abb2-85a03a40a1c9 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:25.042584 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f4d43f-a5d5-46c4-a7ca-fcd4e772f90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:25.043173 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f4d43f-a5d5-46c4-a7ca-fcd4e772f90c admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:25.043521 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f4d43f-a5d5-46c4-a7ca-fcd4e772f90c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:25.043798 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f4d43f-a5d5-46c4-a7ca-fcd4e772f90c admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:25.044410 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f4d43f-a5d5-46c4-a7ca-fcd4e772f90c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:25.046352 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f4d43f-a5d5-46c4-a7ca-fcd4e772f90c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:25.071389 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 617/1251] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:36:25 2022] POST /identity/v3/users => generated 279 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:36:25.071389 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f4d43f-a5d5-46c4-a7ca-fcd4e772f90c admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:25.079111 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f4d43f-a5d5-46c4-a7ca-fcd4e772f90c admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:25.108459 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e071bbe5-146c-429d-abb2-85a03a40a1c9 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:25.114350 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bac246-d7fe-44bc-bca1-95296da806ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:25.114955 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bac246-d7fe-44bc-bca1-95296da806ee admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:25.115232 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bac246-d7fe-44bc-bca1-95296da806ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:25.115504 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bac246-d7fe-44bc-bca1-95296da806ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:25.116127 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bac246-d7fe-44bc-bca1-95296da806ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:25.118057 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bac246-d7fe-44bc-bca1-95296da806ee admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:25.138191 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88de55df-e4c9-427e-aefd-967174d78dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:25.139037 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de55df-e4c9-427e-aefd-967174d78dd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:25.139310 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de55df-e4c9-427e-aefd-967174d78dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:25.139582 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de55df-e4c9-427e-aefd-967174d78dd3 admin admin] PATH_INFO: `/v3/projects/857cfc4b6c254af58e41a22fb02b1286/users/fa529017d91d45f08d641224247aa318/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:25.145520 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 635/1252] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:36:25 2022] POST /identity/v3/projects => generated 403 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:36:25.146317 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bac246-d7fe-44bc-bca1-95296da806ee admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:25.154362 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88de55df-e4c9-427e-aefd-967174d78dd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=857cfc4b6c254af58e41a22fb02b1286, user_id=fa529017d91d45f08d641224247aa318, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:25.158438 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88de55df-e4c9-427e-aefd-967174d78dd3 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:25.170869 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 618/1253] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:36:25 2022] PUT /identity/v3/projects/857cfc4b6c254af58e41a22fb02b1286/users/fa529017d91d45f08d641224247aa318/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:36:25.171645 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88de55df-e4c9-427e-aefd-967174d78dd3 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:25.180154 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:25.207278 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:25.220137 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bac246-d7fe-44bc-bca1-95296da806ee admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:25.236193 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88de55df-e4c9-427e-aefd-967174d78dd3 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:25.240809 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7216b2c-859d-4641-97e1-bbe8c11a781c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:25.241466 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7216b2c-859d-4641-97e1-bbe8c11a781c admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:25.241741 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7216b2c-859d-4641-97e1-bbe8c11a781c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:25.242010 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7216b2c-859d-4641-97e1-bbe8c11a781c admin admin] PATH_INFO: `/v3/projects/4e6f8f816ed2443791470c0402ed87df/users/d0507f846e0a4f26bbeda8524c7fafe0/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:25.256654 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7216b2c-859d-4641-97e1-bbe8c11a781c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e6f8f816ed2443791470c0402ed87df, user_id=d0507f846e0a4f26bbeda8524c7fafe0, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:25.259391 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7216b2c-859d-4641-97e1-bbe8c11a781c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:25.272913 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 619/1254] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:36:25 2022] PUT /identity/v3/projects/4e6f8f816ed2443791470c0402ed87df/users/d0507f846e0a4f26bbeda8524c7fafe0/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:36:25.273535 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7216b2c-859d-4641-97e1-bbe8c11a781c admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:25.283167 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b057a05e-75ed-4a71-b17f-a62f2baa5b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:25.283773 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057a05e-75ed-4a71-b17f-a62f2baa5b34 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:25.284051 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057a05e-75ed-4a71-b17f-a62f2baa5b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:25.284319 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057a05e-75ed-4a71-b17f-a62f2baa5b34 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:25.285137 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b057a05e-75ed-4a71-b17f-a62f2baa5b34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:25.287092 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b057a05e-75ed-4a71-b17f-a62f2baa5b34 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:25.318380 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 636/1255] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:36:25 2022] POST /identity/v3/projects => generated 379 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:36:25.318380 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b057a05e-75ed-4a71-b17f-a62f2baa5b34 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:25.340478 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7216b2c-859d-4641-97e1-bbe8c11a781c admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:25.379355 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b057a05e-75ed-4a71-b17f-a62f2baa5b34 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:25.383963 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c5a438-a1aa-4379-acbe-63fbce2515ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:25.384596 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c5a438-a1aa-4379-acbe-63fbce2515ad admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:25.385002 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c5a438-a1aa-4379-acbe-63fbce2515ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:25.385275 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c5a438-a1aa-4379-acbe-63fbce2515ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:25.385901 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c5a438-a1aa-4379-acbe-63fbce2515ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:25.387890 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c5a438-a1aa-4379-acbe-63fbce2515ad admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:25.396882 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3759ac99-02de-4d02-929b-c06c7cc212b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:25.397492 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759ac99-02de-4d02-929b-c06c7cc212b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:25.397776 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759ac99-02de-4d02-929b-c06c7cc212b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:25.398045 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759ac99-02de-4d02-929b-c06c7cc212b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:25.398669 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3759ac99-02de-4d02-929b-c06c7cc212b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:25.400607 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3759ac99-02de-4d02-929b-c06c7cc212b6 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:25.415795 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 637/1256] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:36:25 2022] POST /identity/v3/projects => generated 393 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:36:25.416726 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c5a438-a1aa-4379-acbe-63fbce2515ad admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:25.430211 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 620/1257] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:36:25 2022] POST /identity/v3/projects => generated 377 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:36:25.431242 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3759ac99-02de-4d02-929b-c06c7cc212b6 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:25.456051 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:25.485128 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c5a438-a1aa-4379-acbe-63fbce2515ad admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:25.497901 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:25.498483 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3759ac99-02de-4d02-929b-c06c7cc212b6 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:25.503310 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c655ab-d76b-4bc1-a4fe-ab350832e134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:25.504030 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c655ab-d76b-4bc1-a4fe-ab350832e134 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:25.504320 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c655ab-d76b-4bc1-a4fe-ab350832e134 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:25.504860 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c655ab-d76b-4bc1-a4fe-ab350832e134 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:25.505505 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c655ab-d76b-4bc1-a4fe-ab350832e134 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:25.507101 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c655ab-d76b-4bc1-a4fe-ab350832e134 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:25.517396 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 621/1258] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:36:25.518157 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c655ab-d76b-4bc1-a4fe-ab350832e134 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:25.528115 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c655ab-d76b-4bc1-a4fe-ab350832e134 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:25.533450 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4bed96-eb6b-4fe2-aac1-bedb796d3947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:25.534103 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4bed96-eb6b-4fe2-aac1-bedb796d3947 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:25.534442 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4bed96-eb6b-4fe2-aac1-bedb796d3947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:25.534773 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4bed96-eb6b-4fe2-aac1-bedb796d3947 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:25.535519 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4bed96-eb6b-4fe2-aac1-bedb796d3947 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:25.537516 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4bed96-eb6b-4fe2-aac1-bedb796d3947 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:25.547569 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8448e8-abe5-4b4f-962e-14cd353a99cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:25.548209 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8448e8-abe5-4b4f-962e-14cd353a99cd admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:25.548479 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8448e8-abe5-4b4f-962e-14cd353a99cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:25.548803 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8448e8-abe5-4b4f-962e-14cd353a99cd admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:25.549425 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8448e8-abe5-4b4f-962e-14cd353a99cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:25.551356 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8448e8-abe5-4b4f-962e-14cd353a99cd admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:25.559325 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 622/1259] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:36:25 2022] POST /identity/v3/users => generated 302 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:36:25.562635 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ae2c3-a490-49af-b9b8-6d3dd6cb8951 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:25.562635 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ae2c3-a490-49af-b9b8-6d3dd6cb8951 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:25.562635 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ae2c3-a490-49af-b9b8-6d3dd6cb8951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:25.573055 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 638/1260] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:36:25 2022] POST /identity/v3/users => generated 295 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:36:25.574692 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14cd7e0-51c0-4f6c-b91d-43dc4cb99588 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:25.574692 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14cd7e0-51c0-4f6c-b91d-43dc4cb99588 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:25.574959 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14cd7e0-51c0-4f6c-b91d-43dc4cb99588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:25.589286 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-af8ae2c3-a490-49af-b9b8-6d3dd6cb8951 None None] MFA Rules not processed for user `fa529017d91d45f08d641224247aa318`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:25.590508 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-af8ae2c3-a490-49af-b9b8-6d3dd6cb8951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:25.604245 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-d14cd7e0-51c0-4f6c-b91d-43dc4cb99588 None None] MFA Rules not processed for user `d0507f846e0a4f26bbeda8524c7fafe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:25.605446 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d14cd7e0-51c0-4f6c-b91d-43dc4cb99588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:25.656464 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 623/1261] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:25.658006 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8ae2c3-a490-49af-b9b8-6d3dd6cb8951 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:25.668108 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8ae2c3-a490-49af-b9b8-6d3dd6cb8951 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:25.669539 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 639/1262] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:25 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:25.670335 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14cd7e0-51c0-4f6c-b91d-43dc4cb99588 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:25.678096 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ebbc86-b024-4d0e-9b8b-576611706f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:25.678702 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ebbc86-b024-4d0e-9b8b-576611706f09 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:25.678982 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ebbc86-b024-4d0e-9b8b-576611706f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:25.679260 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ebbc86-b024-4d0e-9b8b-576611706f09 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:25.679883 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ebbc86-b024-4d0e-9b8b-576611706f09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:25.681833 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ebbc86-b024-4d0e-9b8b-576611706f09 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:25.683293 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14cd7e0-51c0-4f6c-b91d-43dc4cb99588 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:25.688002 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8ef901-8f28-48a1-b148-79e60178ff00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:25.688794 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ef901-8f28-48a1-b148-79e60178ff00 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:25.689104 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ef901-8f28-48a1-b148-79e60178ff00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:25.689485 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ef901-8f28-48a1-b148-79e60178ff00 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:25.690118 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8ef901-8f28-48a1-b148-79e60178ff00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:25.692181 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8ef901-8f28-48a1-b148-79e60178ff00 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:25.709830 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 624/1263] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:36:25 2022] POST /identity/v3/users => generated 290 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:36:25.710455 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ebbc86-b024-4d0e-9b8b-576611706f09 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:25.719321 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 640/1264] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:36:25 2022] POST /identity/v3/users => generated 297 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:36:25.721121 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8ef901-8f28-48a1-b148-79e60178ff00 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:25.726845 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ebbc86-b024-4d0e-9b8b-576611706f09 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:25.730742 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8ef901-8f28-48a1-b148-79e60178ff00 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:25.731480 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428757c1-8ebb-4079-b7c5-c32770243c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:25.732082 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428757c1-8ebb-4079-b7c5-c32770243c13 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:25.732363 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428757c1-8ebb-4079-b7c5-c32770243c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:25.732666 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428757c1-8ebb-4079-b7c5-c32770243c13 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:25.733287 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428757c1-8ebb-4079-b7c5-c32770243c13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:25.735881 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428757c1-8ebb-4079-b7c5-c32770243c13 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:25.736263 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006dca7d-6e54-4265-a52f-47b7415a3872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:25.736905 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006dca7d-6e54-4265-a52f-47b7415a3872 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:25.737200 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006dca7d-6e54-4265-a52f-47b7415a3872 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:25.737455 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006dca7d-6e54-4265-a52f-47b7415a3872 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:25.738281 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006dca7d-6e54-4265-a52f-47b7415a3872 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:25.739837 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006dca7d-6e54-4265-a52f-47b7415a3872 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:25.751140 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 641/1265] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:36:25.755875 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006dca7d-6e54-4265-a52f-47b7415a3872 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:25.760477 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 625/1266] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:36:25 2022] POST /identity/v3/users => generated 289 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:36:25.761442 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428757c1-8ebb-4079-b7c5-c32770243c13 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:25.765802 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006dca7d-6e54-4265-a52f-47b7415a3872 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:25.771179 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1be6a9-8d74-45c4-a4eb-5cdccb325d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:25.771666 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428757c1-8ebb-4079-b7c5-c32770243c13 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:25.771794 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1be6a9-8d74-45c4-a4eb-5cdccb325d15 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:25.772041 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1be6a9-8d74-45c4-a4eb-5cdccb325d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:25.772313 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1be6a9-8d74-45c4-a4eb-5cdccb325d15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:25.773008 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1be6a9-8d74-45c4-a4eb-5cdccb325d15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:25.774588 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1be6a9-8d74-45c4-a4eb-5cdccb325d15 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:25.776237 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d25b54-c123-49cd-88b4-915f6519ff99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:25.776860 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d25b54-c123-49cd-88b4-915f6519ff99 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:25.777135 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d25b54-c123-49cd-88b4-915f6519ff99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:25.777402 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d25b54-c123-49cd-88b4-915f6519ff99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:25.778035 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d25b54-c123-49cd-88b4-915f6519ff99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:25.779611 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d25b54-c123-49cd-88b4-915f6519ff99 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:25.785365 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 642/1267] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:36:25.786139 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1be6a9-8d74-45c4-a4eb-5cdccb325d15 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:25.791344 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 626/1268] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:36:25.792552 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988e620-64ef-44d1-9ad4-4512a479fda7 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:25.792839 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988e620-64ef-44d1-9ad4-4512a479fda7 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:25.793108 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988e620-64ef-44d1-9ad4-4512a479fda7 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:25.794424 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 627/1269] 10.222.0.70 () {54 vars in 961 bytes} [Thu Oct 27 09:36:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:36:25.794967 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0988e620-64ef-44d1-9ad4-4512a479fda7 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:25.801878 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1be6a9-8d74-45c4-a4eb-5cdccb325d15 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:25.806430 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686ec7d3-1655-480f-9bde-0831ca08ce24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:25.807052 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ec7d3-1655-480f-9bde-0831ca08ce24 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:25.807333 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ec7d3-1655-480f-9bde-0831ca08ce24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:25.807605 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ec7d3-1655-480f-9bde-0831ca08ce24 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:25.808226 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686ec7d3-1655-480f-9bde-0831ca08ce24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:25.808549 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0988e620-64ef-44d1-9ad4-4512a479fda7 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:25.810257 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686ec7d3-1655-480f-9bde-0831ca08ce24 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:25.813102 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0824940-397e-45e1-b1d7-ac423d506a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:25.813694 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0824940-397e-45e1-b1d7-ac423d506a9a admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:25.813977 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0824940-397e-45e1-b1d7-ac423d506a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:25.814245 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0824940-397e-45e1-b1d7-ac423d506a9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:25.814869 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0824940-397e-45e1-b1d7-ac423d506a9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:25.816428 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0824940-397e-45e1-b1d7-ac423d506a9a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:25.826606 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 628/1270] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:36:25.827395 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0824940-397e-45e1-b1d7-ac423d506a9a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:25.837169 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0824940-397e-45e1-b1d7-ac423d506a9a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:25.839206 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 643/1271] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:36:25 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:36:25.839889 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686ec7d3-1655-480f-9bde-0831ca08ce24 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:25.867282 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:25.880774 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:25.897064 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b0afe3-c7c0-486c-b22c-9219e0c19b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:25.897672 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b0afe3-c7c0-486c-b22c-9219e0c19b64 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:25.897950 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b0afe3-c7c0-486c-b22c-9219e0c19b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:25.898218 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b0afe3-c7c0-486c-b22c-9219e0c19b64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:25.898847 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b0afe3-c7c0-486c-b22c-9219e0c19b64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:25.900435 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b0afe3-c7c0-486c-b22c-9219e0c19b64 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:25.904761 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686ec7d3-1655-480f-9bde-0831ca08ce24 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:25.910044 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afa3c4a-305e-45e4-8d28-c5eeeb76312e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:25.910625 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa3c4a-305e-45e4-8d28-c5eeeb76312e admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:25.910741 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 629/1272] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:36:25.910929 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa3c4a-305e-45e4-8d28-c5eeeb76312e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:25.911221 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa3c4a-305e-45e4-8d28-c5eeeb76312e admin admin] PATH_INFO: `/v3/projects/ad4158b35c7d4afe948732f0ea817257/users/4cc4ce3e1fa748acaaa5ec8864c6e73f/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:25.911494 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b0afe3-c7c0-486c-b22c-9219e0c19b64 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:25.923312 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b0afe3-c7c0-486c-b22c-9219e0c19b64 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:25.927927 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca17fa6d-c8d3-4c74-8e2b-b4bc66d4c095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:25.928396 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afa3c4a-305e-45e4-8d28-c5eeeb76312e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad4158b35c7d4afe948732f0ea817257, user_id=4cc4ce3e1fa748acaaa5ec8864c6e73f, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:25.928539 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17fa6d-c8d3-4c74-8e2b-b4bc66d4c095 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:25.928795 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17fa6d-c8d3-4c74-8e2b-b4bc66d4c095 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:25.929063 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17fa6d-c8d3-4c74-8e2b-b4bc66d4c095 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:25.929678 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca17fa6d-c8d3-4c74-8e2b-b4bc66d4c095 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:25.931226 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca17fa6d-c8d3-4c74-8e2b-b4bc66d4c095 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:25.931339 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afa3c4a-305e-45e4-8d28-c5eeeb76312e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:25.941478 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 630/1273] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:36:25.942339 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca17fa6d-c8d3-4c74-8e2b-b4bc66d4c095 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:25.944906 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 644/1274] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:36:25 2022] PUT /identity/v3/projects/ad4158b35c7d4afe948732f0ea817257/users/4cc4ce3e1fa748acaaa5ec8864c6e73f/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:36:25.945734 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afa3c4a-305e-45e4-8d28-c5eeeb76312e admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:25.952104 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca17fa6d-c8d3-4c74-8e2b-b4bc66d4c095 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:26.008260 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afa3c4a-305e-45e4-8d28-c5eeeb76312e admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:26.010139 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e32c7c-2b3d-45c7-9d09-cb986259f0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:26.010736 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e32c7c-2b3d-45c7-9d09-cb986259f0cc admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:26.011012 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e32c7c-2b3d-45c7-9d09-cb986259f0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:26.011274 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e32c7c-2b3d-45c7-9d09-cb986259f0cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:26.011942 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e32c7c-2b3d-45c7-9d09-cb986259f0cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:26.012980 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15401fa-55fb-40e5-a731-8f246020046d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:26.013588 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15401fa-55fb-40e5-a731-8f246020046d admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:26.013700 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e32c7c-2b3d-45c7-9d09-cb986259f0cc admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:26.013865 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15401fa-55fb-40e5-a731-8f246020046d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:26.014134 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15401fa-55fb-40e5-a731-8f246020046d admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:26.014766 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15401fa-55fb-40e5-a731-8f246020046d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:26.016923 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15401fa-55fb-40e5-a731-8f246020046d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:26.023517 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 631/1275] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:36:26.024693 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4bca6-5e0b-49f4-b6a1-a7fe61454700 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:26.024977 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4bca6-5e0b-49f4-b6a1-a7fe61454700 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:26.025248 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4bca6-5e0b-49f4-b6a1-a7fe61454700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:26.027141 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 645/1276] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:36:26.027829 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15401fa-55fb-40e5-a731-8f246020046d admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:26.038008 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15401fa-55fb-40e5-a731-8f246020046d admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:26.043073 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2836414b-4113-4eb6-b298-9390f51d7dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:26.043753 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2836414b-4113-4eb6-b298-9390f51d7dde admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:26.043928 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2836414b-4113-4eb6-b298-9390f51d7dde admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:26.044202 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2836414b-4113-4eb6-b298-9390f51d7dde admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:26.044888 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2836414b-4113-4eb6-b298-9390f51d7dde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:26.046457 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2836414b-4113-4eb6-b298-9390f51d7dde admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:26.056423 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 646/1277] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:36:26.057219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2836414b-4113-4eb6-b298-9390f51d7dde admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:26.063033 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-09a4bca6-5e0b-49f4-b6a1-a7fe61454700 None None] MFA Rules not processed for user `dbe4a6e4b6184826b30ac4fc13ad9b1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:26.064256 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-09a4bca6-5e0b-49f4-b6a1-a7fe61454700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:26.066903 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2836414b-4113-4eb6-b298-9390f51d7dde admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:26.071483 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4dd1c3-0c8c-4fd4-b956-abd30589f9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:26.072059 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4dd1c3-0c8c-4fd4-b956-abd30589f9ff admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:26.072336 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4dd1c3-0c8c-4fd4-b956-abd30589f9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:26.072646 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4dd1c3-0c8c-4fd4-b956-abd30589f9ff admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:26.073254 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4dd1c3-0c8c-4fd4-b956-abd30589f9ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:26.075125 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4dd1c3-0c8c-4fd4-b956-abd30589f9ff admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:26.097863 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 647/1278] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:36:26 2022] POST /identity/v3/users => generated 293 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:36:26.098765 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4dd1c3-0c8c-4fd4-b956-abd30589f9ff admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:26.109034 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4dd1c3-0c8c-4fd4-b956-abd30589f9ff admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:26.114087 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b198b574-1822-4165-8dee-e1b23c134aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:26.114678 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b198b574-1822-4165-8dee-e1b23c134aa5 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:26.114958 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b198b574-1822-4165-8dee-e1b23c134aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:26.115228 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b198b574-1822-4165-8dee-e1b23c134aa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:26.115853 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b198b574-1822-4165-8dee-e1b23c134aa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:26.117557 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b198b574-1822-4165-8dee-e1b23c134aa5 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:26.128184 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 648/1279] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:36:26.129217 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b198b574-1822-4165-8dee-e1b23c134aa5 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:26.132768 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 632/1280] 10.222.0.70 () {58 vars in 1058 bytes} [Thu Oct 27 09:36:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:26.134068 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def06ae-abc5-4fad-a014-ed403094e944 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:26.134428 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def06ae-abc5-4fad-a014-ed403094e944 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:26.134707 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def06ae-abc5-4fad-a014-ed403094e944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:26.139289 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b198b574-1822-4165-8dee-e1b23c134aa5 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:26.145574 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a8fe25-66b7-48be-b094-a4f7179a5831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:26.146271 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a8fe25-66b7-48be-b094-a4f7179a5831 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:26.146552 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a8fe25-66b7-48be-b094-a4f7179a5831 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:26.146824 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a8fe25-66b7-48be-b094-a4f7179a5831 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:26.147452 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a8fe25-66b7-48be-b094-a4f7179a5831 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:26.149064 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a8fe25-66b7-48be-b094-a4f7179a5831 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:26.164095 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 649/1281] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:36:26.164095 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a8fe25-66b7-48be-b094-a4f7179a5831 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:26.164342 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-1def06ae-abc5-4fad-a014-ed403094e944 None None] MFA Rules not processed for user `4cc4ce3e1fa748acaaa5ec8864c6e73f`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:26.164679 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1def06ae-abc5-4fad-a014-ed403094e944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:26.174771 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a8fe25-66b7-48be-b094-a4f7179a5831 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:26.179692 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9708ad-8448-44ca-bf7b-b7b99ce60c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:26.180346 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9708ad-8448-44ca-bf7b-b7b99ce60c35 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:26.180651 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9708ad-8448-44ca-bf7b-b7b99ce60c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:26.181181 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9708ad-8448-44ca-bf7b-b7b99ce60c35 admin admin] PATH_INFO: `/v3/projects/81430525775d465bb6b8268d1d587def/users/4ccf46600901461e880a41188a92696f/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:26.185611 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:26.198119 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9708ad-8448-44ca-bf7b-b7b99ce60c35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81430525775d465bb6b8268d1d587def, user_id=4ccf46600901461e880a41188a92696f, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:26.200775 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9708ad-8448-44ca-bf7b-b7b99ce60c35 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:26.215711 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 650/1282] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:36:26 2022] PUT /identity/v3/projects/81430525775d465bb6b8268d1d587def/users/4ccf46600901461e880a41188a92696f/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:36:26.220231 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9708ad-8448-44ca-bf7b-b7b99ce60c35 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:26.237459 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 633/1283] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:26 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:26.238331 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def06ae-abc5-4fad-a014-ed403094e944 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:26.248245 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def06ae-abc5-4fad-a014-ed403094e944 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:26.258788 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:26.280037 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9708ad-8448-44ca-bf7b-b7b99ce60c35 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:26.285463 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f515e4-3b61-4904-bb99-3ff932281fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:26.286135 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f515e4-3b61-4904-bb99-3ff932281fc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:26.286419 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f515e4-3b61-4904-bb99-3ff932281fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:26.286692 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f515e4-3b61-4904-bb99-3ff932281fc1 admin admin] PATH_INFO: `/v3/projects/7295248456c446bbb3f7d2d5953c9360/users/2dea51c29cbd46e8bd1454e10eaac012/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:26.301461 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f515e4-3b61-4904-bb99-3ff932281fc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7295248456c446bbb3f7d2d5953c9360, user_id=2dea51c29cbd46e8bd1454e10eaac012, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:26.304202 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f515e4-3b61-4904-bb99-3ff932281fc1 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:26.308231 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a128800-c04d-490d-b118-d7feb02d32fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:26.308994 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a128800-c04d-490d-b118-d7feb02d32fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:26.309286 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a128800-c04d-490d-b118-d7feb02d32fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:26.309577 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a128800-c04d-490d-b118-d7feb02d32fb admin admin] PATH_INFO: `/v3/projects/a417696418e34d1e9abaa74e3c5627e1/users/6b8fa2a89eee41d09977d7c85c49707b/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:26.317829 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 651/1284] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:36:26 2022] PUT /identity/v3/projects/7295248456c446bbb3f7d2d5953c9360/users/2dea51c29cbd46e8bd1454e10eaac012/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:36:26.318636 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f515e4-3b61-4904-bb99-3ff932281fc1 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:26.325811 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a128800-c04d-490d-b118-d7feb02d32fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a417696418e34d1e9abaa74e3c5627e1, user_id=6b8fa2a89eee41d09977d7c85c49707b, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:26.328495 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a128800-c04d-490d-b118-d7feb02d32fb admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:26.342821 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 634/1285] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:36:26 2022] PUT /identity/v3/projects/a417696418e34d1e9abaa74e3c5627e1/users/6b8fa2a89eee41d09977d7c85c49707b/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:36:26.343583 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a128800-c04d-490d-b118-d7feb02d32fb admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:26.385266 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f515e4-3b61-4904-bb99-3ff932281fc1 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:26.405892 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a128800-c04d-490d-b118-d7feb02d32fb admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:26.410483 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb821eec-fccb-4ca6-8e82-e0911b29452c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:26.411129 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb821eec-fccb-4ca6-8e82-e0911b29452c admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:26.411391 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb821eec-fccb-4ca6-8e82-e0911b29452c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:26.411661 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb821eec-fccb-4ca6-8e82-e0911b29452c admin admin] PATH_INFO: `/v3/projects/df306d37ff634f7eaad80fc3bc65b653/users/20c65ed67cd8403489949c4dea572d01/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:26.426323 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb821eec-fccb-4ca6-8e82-e0911b29452c admin admin] RBAC: Authorizing `identity:create_grant(project_id=df306d37ff634f7eaad80fc3bc65b653, user_id=20c65ed67cd8403489949c4dea572d01, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:26.429069 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb821eec-fccb-4ca6-8e82-e0911b29452c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:26.442796 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 635/1286] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:36:26 2022] PUT /identity/v3/projects/df306d37ff634f7eaad80fc3bc65b653/users/20c65ed67cd8403489949c4dea572d01/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:36:26.442954 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bb7d53-f3c5-477a-9d7f-54714c81c143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:26.443573 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb7d53-f3c5-477a-9d7f-54714c81c143 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:26.443868 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb7d53-f3c5-477a-9d7f-54714c81c143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:26.443987 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb821eec-fccb-4ca6-8e82-e0911b29452c admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:26.444129 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb7d53-f3c5-477a-9d7f-54714c81c143 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:26.444830 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bb7d53-f3c5-477a-9d7f-54714c81c143 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:26.446771 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bb7d53-f3c5-477a-9d7f-54714c81c143 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:26.458383 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 652/1287] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:36:26.459230 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bb7d53-f3c5-477a-9d7f-54714c81c143 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:26.469448 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bb7d53-f3c5-477a-9d7f-54714c81c143 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:26.476807 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:26.518942 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb821eec-fccb-4ca6-8e82-e0911b29452c admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:26.524718 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb6042-dd55-4b45-ae1f-e93ecc85f5ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe4a6e4b6184826b30ac4fc13ad9b1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:26.525317 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb6042-dd55-4b45-ae1f-e93ecc85f5ae service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:26.525590 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb6042-dd55-4b45-ae1f-e93ecc85f5ae service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:26.525857 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb6042-dd55-4b45-ae1f-e93ecc85f5ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:26.527386 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c669ed-d53e-46f7-95e4-4f859c9ffbd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:26.528022 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c669ed-d53e-46f7-95e4-4f859c9ffbd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:26.528297 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c669ed-d53e-46f7-95e4-4f859c9ffbd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:26.528616 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c669ed-d53e-46f7-95e4-4f859c9ffbd5 admin admin] PATH_INFO: `/v3/projects/71d1c4930a1b487eb3dca8fde6ce424a/users/286df4c587d7408d988c4739f4ff923a/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:26.539339 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:26.543223 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c669ed-d53e-46f7-95e4-4f859c9ffbd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71d1c4930a1b487eb3dca8fde6ce424a, user_id=286df4c587d7408d988c4739f4ff923a, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:26.546501 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c669ed-d53e-46f7-95e4-4f859c9ffbd5 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:26.561255 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 653/1288] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:36:26 2022] PUT /identity/v3/projects/71d1c4930a1b487eb3dca8fde6ce424a/users/286df4c587d7408d988c4739f4ff923a/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:36:26.562746 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f6f81a-3ad6-435d-95b4-3cdaa313b29b None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:26.563734 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f6f81a-3ad6-435d-95b4-3cdaa313b29b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:26.564054 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f6f81a-3ad6-435d-95b4-3cdaa313b29b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:26.583054 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cb6042-dd55-4b45-ae1f-e93ecc85f5ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:26.584541 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-12f6f81a-3ad6-435d-95b4-3cdaa313b29b None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:26.584664 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cb6042-dd55-4b45-ae1f-e93ecc85f5ae service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:26.585740 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-12f6f81a-3ad6-435d-95b4-3cdaa313b29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:26.594055 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 636/1289] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:36:26 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:36:26.594931 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb6042-dd55-4b45-ae1f-e93ecc85f5ae service nova] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:26.604663 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb6042-dd55-4b45-ae1f-e93ecc85f5ae service nova] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:26.646301 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 654/1290] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:26.650829 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ad2a3-b406-4688-8658-82d5d941400d None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:26.651118 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ad2a3-b406-4688-8658-82d5d941400d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:26.651394 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ad2a3-b406-4688-8658-82d5d941400d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:26.666736 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b638b8-fecd-4db5-af5c-c2d40dfdc679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:26.667428 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b638b8-fecd-4db5-af5c-c2d40dfdc679 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:26.667776 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b638b8-fecd-4db5-af5c-c2d40dfdc679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:26.668116 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b638b8-fecd-4db5-af5c-c2d40dfdc679 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:26.668854 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b638b8-fecd-4db5-af5c-c2d40dfdc679 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:26.670833 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b638b8-fecd-4db5-af5c-c2d40dfdc679 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:26.671240 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-a27ad2a3-b406-4688-8658-82d5d941400d None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:26.672434 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a27ad2a3-b406-4688-8658-82d5d941400d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:26.683225 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 655/1291] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:26.683225 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d655edd-8d95-45a9-8241-0e2d63071b18 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:26.683225 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d655edd-8d95-45a9-8241-0e2d63071b18 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:26.683225 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d655edd-8d95-45a9-8241-0e2d63071b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:26.703502 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 637/1292] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:36:26 2022] POST /identity/v3/projects => generated 357 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:36:26.703502 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe104b-2c40-40be-bb2e-8e3f3a6c7ac0 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:26.703502 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe104b-2c40-40be-bb2e-8e3f3a6c7ac0 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:26.703502 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe104b-2c40-40be-bb2e-8e3f3a6c7ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:26.705759 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-1d655edd-8d95-45a9-8241-0e2d63071b18 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:26.707253 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1d655edd-8d95-45a9-8241-0e2d63071b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:26.724302 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-ccbe104b-2c40-40be-bb2e-8e3f3a6c7ac0 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:26.725909 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ccbe104b-2c40-40be-bb2e-8e3f3a6c7ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:26.769863 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 656/1293] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:26.774116 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d655edd-8d95-45a9-8241-0e2d63071b18 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:26.777530 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:26.784226 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d655edd-8d95-45a9-8241-0e2d63071b18 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:26.788931 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882d5e55-6a77-47c2-91e6-69540c05f5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:26.789605 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882d5e55-6a77-47c2-91e6-69540c05f5fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:26.789885 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882d5e55-6a77-47c2-91e6-69540c05f5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:26.790157 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882d5e55-6a77-47c2-91e6-69540c05f5fc admin admin] PATH_INFO: `/v3/projects/5efbf0eeb65d47efa636377c34e2f3d6/users/5994e18e32ae40f394e8971084d913b9/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:26.795256 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 638/1294] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:26.798884 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790a9ee-ee8e-4648-b429-d033586790ee None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:26.799607 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790a9ee-ee8e-4648-b429-d033586790ee None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:26.800775 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790a9ee-ee8e-4648-b429-d033586790ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:26.807280 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882d5e55-6a77-47c2-91e6-69540c05f5fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5efbf0eeb65d47efa636377c34e2f3d6, user_id=5994e18e32ae40f394e8971084d913b9, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:26.810610 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882d5e55-6a77-47c2-91e6-69540c05f5fc admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:26.822317 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-f790a9ee-ee8e-4648-b429-d033586790ee None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:26.823575 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f790a9ee-ee8e-4648-b429-d033586790ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:26.828595 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 657/1295] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:36:26 2022] PUT /identity/v3/projects/5efbf0eeb65d47efa636377c34e2f3d6/users/5994e18e32ae40f394e8971084d913b9/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:36:26.828595 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882d5e55-6a77-47c2-91e6-69540c05f5fc admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:26.886083 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882d5e55-6a77-47c2-91e6-69540c05f5fc admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:26.887302 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 639/1296] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:26.888131 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f790a9ee-ee8e-4648-b429-d033586790ee None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:26.891636 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f790a9ee-ee8e-4648-b429-d033586790ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:26.893988 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e90776-c507-4eb3-bee3-065cc6bfc4d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:26.894903 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e90776-c507-4eb3-bee3-065cc6bfc4d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:26.895362 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e90776-c507-4eb3-bee3-065cc6bfc4d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:26.895792 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e90776-c507-4eb3-bee3-065cc6bfc4d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:26.903148 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f790a9ee-ee8e-4648-b429-d033586790ee None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:26.905210 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e90776-c507-4eb3-bee3-065cc6bfc4d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:26.906636 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e90776-c507-4eb3-bee3-065cc6bfc4d7 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:26.906916 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c41e99b-40a0-4ad1-be68-2d78bbe1903f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:26.907508 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41e99b-40a0-4ad1-be68-2d78bbe1903f service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:26.907782 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41e99b-40a0-4ad1-be68-2d78bbe1903f service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:26.908052 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41e99b-40a0-4ad1-be68-2d78bbe1903f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:26.916008 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 658/1297] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:36:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:36:26.916857 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e90776-c507-4eb3-bee3-065cc6bfc4d7 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:26.917656 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c41e99b-40a0-4ad1-be68-2d78bbe1903f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:26.919424 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c41e99b-40a0-4ad1-be68-2d78bbe1903f service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:26.921132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:26.928039 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e90776-c507-4eb3-bee3-065cc6bfc4d7 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:26.933289 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4545fe99-b07b-426d-9a66-610f668e3fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:26.933828 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4545fe99-b07b-426d-9a66-610f668e3fa1 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:26.934100 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4545fe99-b07b-426d-9a66-610f668e3fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:26.934364 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4545fe99-b07b-426d-9a66-610f668e3fa1 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:26.935038 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 640/1298] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:36:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:36:26.935175 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4545fe99-b07b-426d-9a66-610f668e3fa1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:26.936011 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c41e99b-40a0-4ad1-be68-2d78bbe1903f service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:26.937174 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4545fe99-b07b-426d-9a66-610f668e3fa1 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:26.945862 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c41e99b-40a0-4ad1-be68-2d78bbe1903f service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:26.950587 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63811996-c05b-4205-ade1-450f791f7e0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:26.951396 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63811996-c05b-4205-ade1-450f791f7e0c service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:26.951697 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63811996-c05b-4205-ade1-450f791f7e0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:26.952197 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63811996-c05b-4205-ade1-450f791f7e0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:26.963745 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63811996-c05b-4205-ade1-450f791f7e0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:26.965122 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63811996-c05b-4205-ade1-450f791f7e0c service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:26.966350 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 659/1299] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:36:26 2022] POST /identity/v3/users => generated 285 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:36:26.967455 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4545fe99-b07b-426d-9a66-610f668e3fa1 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:26.976842 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 641/1300] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:36:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:36:26.977575 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63811996-c05b-4205-ade1-450f791f7e0c service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:26.982970 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4545fe99-b07b-426d-9a66-610f668e3fa1 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:26.989613 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e67bb3-1e26-4271-90f3-709c77c23e8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:26.989999 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63811996-c05b-4205-ade1-450f791f7e0c service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:26.990694 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e67bb3-1e26-4271-90f3-709c77c23e8c service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:26.991148 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e67bb3-1e26-4271-90f3-709c77c23e8c service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:26.991483 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e67bb3-1e26-4271-90f3-709c77c23e8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:26.995844 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7986cdff-84f6-4809-b854-afd3cecad9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:26.996667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7986cdff-84f6-4809-b854-afd3cecad9bb admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:26.996949 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7986cdff-84f6-4809-b854-afd3cecad9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:26.997653 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7986cdff-84f6-4809-b854-afd3cecad9bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:26.998357 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7986cdff-84f6-4809-b854-afd3cecad9bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:27.000716 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7986cdff-84f6-4809-b854-afd3cecad9bb admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:27.005249 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e67bb3-1e26-4271-90f3-709c77c23e8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:27.007160 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e67bb3-1e26-4271-90f3-709c77c23e8c service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:27.014728 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 642/1301] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:36:27.016024 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0702999-6388-407c-b201-6b3c90915b24 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:27.016642 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0702999-6388-407c-b201-6b3c90915b24 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:27.016642 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0702999-6388-407c-b201-6b3c90915b24 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:27.018333 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 643/1302] 10.222.0.70 () {54 vars in 912 bytes} [Thu Oct 27 09:36:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:36:27.019064 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0702999-6388-407c-b201-6b3c90915b24 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:27.021517 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 660/1303] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:36:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:36:27.022902 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e67bb3-1e26-4271-90f3-709c77c23e8c service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:27.033946 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0702999-6388-407c-b201-6b3c90915b24 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:27.035437 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e67bb3-1e26-4271-90f3-709c77c23e8c service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:27.038530 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856ca781-2dfd-4e43-a471-b2f55d3e7832 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:27.039401 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856ca781-2dfd-4e43-a471-b2f55d3e7832 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:27.039834 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856ca781-2dfd-4e43-a471-b2f55d3e7832 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:27.040262 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856ca781-2dfd-4e43-a471-b2f55d3e7832 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:27.043238 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5295eee-b5c2-4365-a458-dec7c7bdc1c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:27.044315 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5295eee-b5c2-4365-a458-dec7c7bdc1c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:27.044750 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5295eee-b5c2-4365-a458-dec7c7bdc1c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:27.045183 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5295eee-b5c2-4365-a458-dec7c7bdc1c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:27.046249 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5295eee-b5c2-4365-a458-dec7c7bdc1c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:27.049246 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5295eee-b5c2-4365-a458-dec7c7bdc1c4 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:27.052817 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856ca781-2dfd-4e43-a471-b2f55d3e7832 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:27.054486 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856ca781-2dfd-4e43-a471-b2f55d3e7832 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:27.066099 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 661/1304] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:36:27 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:36:27.067541 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5295eee-b5c2-4365-a458-dec7c7bdc1c4 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:27.068320 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 644/1305] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:36:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:36:27.069966 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e0ba89-b2f2-4bc1-98c8-22dce10a9cad None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:27.070400 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e0ba89-b2f2-4bc1-98c8-22dce10a9cad None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:27.070902 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e0ba89-b2f2-4bc1-98c8-22dce10a9cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:27.086817 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5295eee-b5c2-4365-a458-dec7c7bdc1c4 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:27.093495 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5e94e7-e9a7-4d62-ba6e-fc654e5b23c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:27.095107 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e94e7-e9a7-4d62-ba6e-fc654e5b23c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:27.095424 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e94e7-e9a7-4d62-ba6e-fc654e5b23c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:27.095798 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e94e7-e9a7-4d62-ba6e-fc654e5b23c8 admin admin] PATH_INFO: `/v3/projects/5efbf0eeb65d47efa636377c34e2f3d6/users/5994e18e32ae40f394e8971084d913b9/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:27.098169 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5e94e7-e9a7-4d62-ba6e-fc654e5b23c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5efbf0eeb65d47efa636377c34e2f3d6, user_id=5994e18e32ae40f394e8971084d913b9, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:27.102286 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5e94e7-e9a7-4d62-ba6e-fc654e5b23c8 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:27.104186 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a6e0ba89-b2f2-4bc1-98c8-22dce10a9cad None None] MFA Rules not processed for user `dbe4a6e4b6184826b30ac4fc13ad9b1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:27.105800 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a6e0ba89-b2f2-4bc1-98c8-22dce10a9cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:27.119212 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 662/1306] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:36:27 2022] PUT /identity/v3/projects/5efbf0eeb65d47efa636377c34e2f3d6/users/5994e18e32ae40f394e8971084d913b9/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:36:27.120147 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5e94e7-e9a7-4d62-ba6e-fc654e5b23c8 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:27.199576 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 645/1307] 10.222.0.70 () {58 vars in 1009 bytes} [Thu Oct 27 09:36:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:27.201037 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb8365-0059-4650-9270-16bb7f86a6b1 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:27.201536 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb8365-0059-4650-9270-16bb7f86a6b1 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:27.201853 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb8365-0059-4650-9270-16bb7f86a6b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:27.231460 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5e94e7-e9a7-4d62-ba6e-fc654e5b23c8 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:27.234353 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-f3fb8365-0059-4650-9270-16bb7f86a6b1 None None] MFA Rules not processed for user `5994e18e32ae40f394e8971084d913b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:27.236631 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f3fb8365-0059-4650-9270-16bb7f86a6b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:27.238082 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b6e49-e215-4adb-9c8c-80cd74ecb487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:27.238759 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b6e49-e215-4adb-9c8c-80cd74ecb487 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:27.239105 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b6e49-e215-4adb-9c8c-80cd74ecb487 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:27.239448 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b6e49-e215-4adb-9c8c-80cd74ecb487 admin admin] PATH_INFO: `/v3/projects/7a28f38950ad4721bc14a6a805e1ce9c/users/54970c6f86c14078af579866d8e046e0/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:27.259581 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389b6e49-e215-4adb-9c8c-80cd74ecb487 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a28f38950ad4721bc14a6a805e1ce9c, user_id=54970c6f86c14078af579866d8e046e0, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:27.262529 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:27.264277 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389b6e49-e215-4adb-9c8c-80cd74ecb487 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:27.287876 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 663/1308] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:36:27 2022] PUT /identity/v3/projects/7a28f38950ad4721bc14a6a805e1ce9c/users/54970c6f86c14078af579866d8e046e0/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:36:27.292268 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b6e49-e215-4adb-9c8c-80cd74ecb487 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:27.341638 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 646/1309] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:27 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:27.344911 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb8365-0059-4650-9270-16bb7f86a6b1 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:27.377124 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb8365-0059-4650-9270-16bb7f86a6b1 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:27.431236 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b6e49-e215-4adb-9c8c-80cd74ecb487 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:27.442240 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34943bc-c53f-4e49-8647-50bf1fd19582 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:27.444757 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34943bc-c53f-4e49-8647-50bf1fd19582 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:27.446146 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34943bc-c53f-4e49-8647-50bf1fd19582 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:27.447852 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34943bc-c53f-4e49-8647-50bf1fd19582 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:27.468462 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205052b3-77ab-4c94-a0b9-69d07549d935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:27.470011 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205052b3-77ab-4c94-a0b9-69d07549d935 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:27.470493 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205052b3-77ab-4c94-a0b9-69d07549d935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:27.470771 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205052b3-77ab-4c94-a0b9-69d07549d935 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:27.471597 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205052b3-77ab-4c94-a0b9-69d07549d935 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:27.473750 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205052b3-77ab-4c94-a0b9-69d07549d935 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:27.496194 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 647/1310] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:36:27 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:36:27.497355 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205052b3-77ab-4c94-a0b9-69d07549d935 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:27.524562 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205052b3-77ab-4c94-a0b9-69d07549d935 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:27.534899 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb6f20-78d1-45ef-984d-d39216f2ca75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:27.535521 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb6f20-78d1-45ef-984d-d39216f2ca75 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:27.535803 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb6f20-78d1-45ef-984d-d39216f2ca75 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:27.536065 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb6f20-78d1-45ef-984d-d39216f2ca75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:27.561416 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:27.610395 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34943bc-c53f-4e49-8647-50bf1fd19582 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:27.612160 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34943bc-c53f-4e49-8647-50bf1fd19582 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:27.629043 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 664/1311] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:36:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:36:27.630247 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34943bc-c53f-4e49-8647-50bf1fd19582 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:27.643608 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbb6f20-78d1-45ef-984d-d39216f2ca75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:27.645189 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbb6f20-78d1-45ef-984d-d39216f2ca75 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:27.645325 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34943bc-c53f-4e49-8647-50bf1fd19582 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:27.652666 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e0a009-f26b-4c5f-89cf-d282b4437814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:27.653464 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e0a009-f26b-4c5f-89cf-d282b4437814 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:27.653769 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e0a009-f26b-4c5f-89cf-d282b4437814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:27.654048 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e0a009-f26b-4c5f-89cf-d282b4437814 admin admin] PATH_INFO: `/v3/projects/7a28f38950ad4721bc14a6a805e1ce9c/users/54970c6f86c14078af579866d8e046e0/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:27.656699 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e0a009-f26b-4c5f-89cf-d282b4437814 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a28f38950ad4721bc14a6a805e1ce9c, user_id=54970c6f86c14078af579866d8e046e0, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:27.661082 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e0a009-f26b-4c5f-89cf-d282b4437814 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:27.670561 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 648/1312] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:36:27 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:36:27.723218 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea34150-498b-44dc-97a7-5e58134f548e None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:27.723761 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea34150-498b-44dc-97a7-5e58134f548e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:27.724461 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 665/1313] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:36:27 2022] PUT /identity/v3/projects/7a28f38950ad4721bc14a6a805e1ce9c/users/54970c6f86c14078af579866d8e046e0/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:36:27.724639 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea34150-498b-44dc-97a7-5e58134f548e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:27.779354 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-1ea34150-498b-44dc-97a7-5e58134f548e None None] MFA Rules not processed for user `54970c6f86c14078af579866d8e046e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:27.781443 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1ea34150-498b-44dc-97a7-5e58134f548e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:27.862124 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e267cf-86e5-434b-8ad7-996aedd065ed None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:27.862847 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e267cf-86e5-434b-8ad7-996aedd065ed None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:27.863130 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e267cf-86e5-434b-8ad7-996aedd065ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:27.926735 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:27.928975 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-61e267cf-86e5-434b-8ad7-996aedd065ed None None] MFA Rules not processed for user `4ccf46600901461e880a41188a92696f`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:27.930245 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-61e267cf-86e5-434b-8ad7-996aedd065ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:27.998515 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 649/1314] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:27 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:36:27.999695 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea34150-498b-44dc-97a7-5e58134f548e None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:28.121686 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 666/1315] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:27 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:36:28.127883 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e267cf-86e5-434b-8ad7-996aedd065ed None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:28.158821 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea34150-498b-44dc-97a7-5e58134f548e None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:28.176475 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205af7ec-a25b-4188-a4ce-e232625b6ae8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:28.177157 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205af7ec-a25b-4188-a4ce-e232625b6ae8 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:28.177561 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205af7ec-a25b-4188-a4ce-e232625b6ae8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:28.177839 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205af7ec-a25b-4188-a4ce-e232625b6ae8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:28.224505 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e267cf-86e5-434b-8ad7-996aedd065ed None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:28.230261 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71182961-b7a8-495c-8ce2-9d4482877e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:28.231668 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71182961-b7a8-495c-8ce2-9d4482877e6a admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:28.231993 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71182961-b7a8-495c-8ce2-9d4482877e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:28.232423 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71182961-b7a8-495c-8ce2-9d4482877e6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:28.233239 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71182961-b7a8-495c-8ce2-9d4482877e6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:28.235949 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71182961-b7a8-495c-8ce2-9d4482877e6a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:28.286981 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 667/1316] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:36:28 2022] POST /identity/v3/projects => generated 403 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:36:28.290716 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71182961-b7a8-495c-8ce2-9d4482877e6a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:28.319225 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205af7ec-a25b-4188-a4ce-e232625b6ae8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:28.321153 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205af7ec-a25b-4188-a4ce-e232625b6ae8 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:28.339210 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 650/1317] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:36:27 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:36:28.374063 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:28.434846 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71182961-b7a8-495c-8ce2-9d4482877e6a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:28.458546 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c24613-67b2-484f-a64c-f85a0aa05019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:28.459296 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c24613-67b2-484f-a64c-f85a0aa05019 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:28.460095 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c24613-67b2-484f-a64c-f85a0aa05019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:28.460446 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c24613-67b2-484f-a64c-f85a0aa05019 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:28.461194 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c24613-67b2-484f-a64c-f85a0aa05019 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:28.463217 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c24613-67b2-484f-a64c-f85a0aa05019 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:28.513616 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205af7ec-a25b-4188-a4ce-e232625b6ae8 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:28.534478 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205af7ec-a25b-4188-a4ce-e232625b6ae8 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:28.549896 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 668/1318] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:36:28 2022] POST /identity/v3/users => generated 308 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:36:28.562420 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c24613-67b2-484f-a64c-f85a0aa05019 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:28.585546 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c24613-67b2-484f-a64c-f85a0aa05019 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:28.606172 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f94d66-4f75-46b6-aba4-bce6870db3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:28.606809 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f94d66-4f75-46b6-aba4-bce6870db3a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:28.607092 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f94d66-4f75-46b6-aba4-bce6870db3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:28.609485 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f94d66-4f75-46b6-aba4-bce6870db3a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:28.610316 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f94d66-4f75-46b6-aba4-bce6870db3a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:28.615633 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f94d66-4f75-46b6-aba4-bce6870db3a7 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:28.657437 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 669/1319] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:36:28.661498 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f94d66-4f75-46b6-aba4-bce6870db3a7 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:28.691798 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f94d66-4f75-46b6-aba4-bce6870db3a7 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:28.696983 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cbb70c-4854-4e48-8229-fe7647f0e798 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:28.698076 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cbb70c-4854-4e48-8229-fe7647f0e798 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:28.698278 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cbb70c-4854-4e48-8229-fe7647f0e798 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:28.698553 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cbb70c-4854-4e48-8229-fe7647f0e798 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:28.709033 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d9f3d-fad9-442f-93c3-bf54daaaa1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:28.709714 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d9f3d-fad9-442f-93c3-bf54daaaa1d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:28.709994 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d9f3d-fad9-442f-93c3-bf54daaaa1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:28.710265 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d9f3d-fad9-442f-93c3-bf54daaaa1d2 admin admin] PATH_INFO: `/v3/projects/19c0e11ecca94dcbb9eaace33e413f9f/users/c038d9499edf46e59c1ba7b9a00758e9/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:28.744947 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8d9f3d-fad9-442f-93c3-bf54daaaa1d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19c0e11ecca94dcbb9eaace33e413f9f, user_id=c038d9499edf46e59c1ba7b9a00758e9, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:28.747859 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8d9f3d-fad9-442f-93c3-bf54daaaa1d2 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:28.776488 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 670/1320] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:36:28 2022] PUT /identity/v3/projects/19c0e11ecca94dcbb9eaace33e413f9f/users/c038d9499edf46e59c1ba7b9a00758e9/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:36:28.780457 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d9f3d-fad9-442f-93c3-bf54daaaa1d2 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:28.796331 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:28.842753 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cbb70c-4854-4e48-8229-fe7647f0e798 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:28.844583 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cbb70c-4854-4e48-8229-fe7647f0e798 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:28.884510 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 651/1321] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:36:28 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:36:28.934099 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d9f3d-fad9-442f-93c3-bf54daaaa1d2 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:28.957864 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47aa865-f4a4-482d-a22e-97b6a29263f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:28.958766 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47aa865-f4a4-482d-a22e-97b6a29263f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:28.960751 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47aa865-f4a4-482d-a22e-97b6a29263f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:28.961006 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47aa865-f4a4-482d-a22e-97b6a29263f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:28.963337 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47aa865-f4a4-482d-a22e-97b6a29263f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:28.965253 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47aa865-f4a4-482d-a22e-97b6a29263f0 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:28.996533 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cbb70c-4854-4e48-8229-fe7647f0e798 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:29.005837 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 671/1322] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 09:36:29.013891 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cbb70c-4854-4e48-8229-fe7647f0e798 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:29.024628 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3be0f2-6de4-49ec-9047-790cd67580cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:29.026902 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3be0f2-6de4-49ec-9047-790cd67580cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:29.027523 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3be0f2-6de4-49ec-9047-790cd67580cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:29.028049 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3be0f2-6de4-49ec-9047-790cd67580cb admin admin] PATH_INFO: `/v3/projects/19c0e11ecca94dcbb9eaace33e413f9f/users/c038d9499edf46e59c1ba7b9a00758e9/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:29.033659 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3be0f2-6de4-49ec-9047-790cd67580cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=19c0e11ecca94dcbb9eaace33e413f9f, user_id=c038d9499edf46e59c1ba7b9a00758e9, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:29.038808 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3be0f2-6de4-49ec-9047-790cd67580cb admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:29.071753 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47aa865-f4a4-482d-a22e-97b6a29263f0 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:29.086289 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 652/1323] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:36:28 2022] PUT /identity/v3/projects/19c0e11ecca94dcbb9eaace33e413f9f/users/c038d9499edf46e59c1ba7b9a00758e9/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:36:29.226801 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47aa865-f4a4-482d-a22e-97b6a29263f0 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:29.330700 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:29.354087 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be9ce8c-185a-4f96-b2b4-cdae4428b085 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:29.354727 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be9ce8c-185a-4f96-b2b4-cdae4428b085 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:29.355004 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be9ce8c-185a-4f96-b2b4-cdae4428b085 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:29.355276 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be9ce8c-185a-4f96-b2b4-cdae4428b085 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:29.399594 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4303a03-3122-4c84-8407-95f12751603c None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:29.400182 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4303a03-3122-4c84-8407-95f12751603c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:29.400463 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4303a03-3122-4c84-8407-95f12751603c None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:29.402439 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 653/1324] 10.222.0.70 () {54 vars in 961 bytes} [Thu Oct 27 09:36:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:36:29.410458 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc46e985-f702-4548-9367-176c5b5dd375 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:29.410674 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc46e985-f702-4548-9367-176c5b5dd375 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:29.410904 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc46e985-f702-4548-9367-176c5b5dd375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:29.454271 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-dc46e985-f702-4548-9367-176c5b5dd375 None None] MFA Rules not processed for user `dbe4a6e4b6184826b30ac4fc13ad9b1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:29.460041 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-dc46e985-f702-4548-9367-176c5b5dd375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:29.508833 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be9ce8c-185a-4f96-b2b4-cdae4428b085 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:29.509942 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be9ce8c-185a-4f96-b2b4-cdae4428b085 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:29.533254 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 672/1325] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:36:29 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:36:29.585461 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 654/1326] 10.222.0.70 () {58 vars in 1058 bytes} [Thu Oct 27 09:36:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:36:29.591677 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0886352-6d42-416c-81c0-d5ff39059e9f None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:29.592163 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0886352-6d42-416c-81c0-d5ff39059e9f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:29.592545 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0886352-6d42-416c-81c0-d5ff39059e9f None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:29.594014 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 673/1327] 10.222.0.70 () {54 vars in 912 bytes} [Thu Oct 27 09:36:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:36:29.600562 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa6dd7-3e81-48ec-b736-2f334596d0f9 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:29.600917 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa6dd7-3e81-48ec-b736-2f334596d0f9 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:29.601307 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa6dd7-3e81-48ec-b736-2f334596d0f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:29.642449 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-53aa6dd7-3e81-48ec-b736-2f334596d0f9 None None] MFA Rules not processed for user `dbe4a6e4b6184826b30ac4fc13ad9b1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:29.644115 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-53aa6dd7-3e81-48ec-b736-2f334596d0f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:29.668174 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 655/1328] 10.222.0.70 () {58 vars in 1009 bytes} [Thu Oct 27 09:36:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:29.682125 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0886352-6d42-416c-81c0-d5ff39059e9f None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:29.703655 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0886352-6d42-416c-81c0-d5ff39059e9f None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:29.710021 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871c764a-624c-4a6e-b368-9c6b07ecf1df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:29.710632 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c764a-624c-4a6e-b368-9c6b07ecf1df service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:29.711015 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c764a-624c-4a6e-b368-9c6b07ecf1df service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:29.711557 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c764a-624c-4a6e-b368-9c6b07ecf1df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:29.748489 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871c764a-624c-4a6e-b368-9c6b07ecf1df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:29.750006 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871c764a-624c-4a6e-b368-9c6b07ecf1df service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:29.792762 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 674/1329] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:36:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:36:32.815386 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04724d56-187a-4ae6-9e39-595ba5f73a2d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:32.816542 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04724d56-187a-4ae6-9e39-595ba5f73a2d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:32.817007 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04724d56-187a-4ae6-9e39-595ba5f73a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:32.884610 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-04724d56-187a-4ae6-9e39-595ba5f73a2d None None] MFA Rules not processed for user `c038d9499edf46e59c1ba7b9a00758e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:32.886025 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-04724d56-187a-4ae6-9e39-595ba5f73a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:33.034802 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 656/1330] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:32 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:36.388182 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ab6f9-f2ed-4ea6-a2ef-2edb0a498df4 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:36.389240 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ab6f9-f2ed-4ea6-a2ef-2edb0a498df4 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:36.389742 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ab6f9-f2ed-4ea6-a2ef-2edb0a498df4 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:36.391405 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 675/1331] 10.222.0.70 () {54 vars in 921 bytes} [Thu Oct 27 09:36:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:36:36.406112 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d04f3-4c00-4af1-892a-4329594117da None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:36.407749 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d04f3-4c00-4af1-892a-4329594117da None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:36.408241 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d04f3-4c00-4af1-892a-4329594117da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:36.455647 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-4d0d04f3-4c00-4af1-892a-4329594117da None None] MFA Rules not processed for user `2316b30a6e7b4175bd7b90583584eab5`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:36.457346 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4d0d04f3-4c00-4af1-892a-4329594117da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:36.534076 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:36.641736 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 657/1332] 10.222.0.70 () {58 vars in 1018 bytes} [Thu Oct 27 09:36:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:36:36.687815 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ab6f9-f2ed-4ea6-a2ef-2edb0a498df4 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:36.721502 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ab6f9-f2ed-4ea6-a2ef-2edb0a498df4 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:36.743081 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df62189d-6b93-476b-ad56-9223e7f3ea72 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2316b30a6e7b4175bd7b90583584eab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:36.743846 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62189d-6b93-476b-ad56-9223e7f3ea72 service placement] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:36.746546 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62189d-6b93-476b-ad56-9223e7f3ea72 service placement] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:36.746966 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62189d-6b93-476b-ad56-9223e7f3ea72 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:36.767127 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df62189d-6b93-476b-ad56-9223e7f3ea72 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:36.771992 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df62189d-6b93-476b-ad56-9223e7f3ea72 service placement] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:36.803761 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 676/1333] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:36:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:36:39.131375 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e9fc8-cdca-454b-bb40-9ad7814ba185 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:39.131954 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e9fc8-cdca-454b-bb40-9ad7814ba185 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:39.131954 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e9fc8-cdca-454b-bb40-9ad7814ba185 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:39.133787 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 658/1334] 10.222.0.70 () {54 vars in 921 bytes} [Thu Oct 27 09:36:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:36:39.247065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b45cd9f-3044-43e6-b354-72cf771ad2cc None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:39.247581 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b45cd9f-3044-43e6-b354-72cf771ad2cc None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:39.247824 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b45cd9f-3044-43e6-b354-72cf771ad2cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:39.283294 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e9fc8-cdca-454b-bb40-9ad7814ba185 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:39.304247 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-4b45cd9f-3044-43e6-b354-72cf771ad2cc None None] MFA Rules not processed for user `2316b30a6e7b4175bd7b90583584eab5`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:39.308168 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4b45cd9f-3044-43e6-b354-72cf771ad2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:39.355211 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 677/1335] 10.222.0.70 () {58 vars in 1018 bytes} [Thu Oct 27 09:36:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:39.405549 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b45cd9f-3044-43e6-b354-72cf771ad2cc None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:39.420970 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b45cd9f-3044-43e6-b354-72cf771ad2cc None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:39.438055 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e9fc8-cdca-454b-bb40-9ad7814ba185 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:39.439260 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c47a2-1da1-44e8-b8f0-30c2afb02d2b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2316b30a6e7b4175bd7b90583584eab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:39.441286 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c47a2-1da1-44e8-b8f0-30c2afb02d2b service placement] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:39.443088 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c47a2-1da1-44e8-b8f0-30c2afb02d2b service placement] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:39.443439 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c47a2-1da1-44e8-b8f0-30c2afb02d2b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:39.449657 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cdde1c-1d3d-4dc1-94fb-0cdc32a30916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:39.450596 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cdde1c-1d3d-4dc1-94fb-0cdc32a30916 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:39.453338 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cdde1c-1d3d-4dc1-94fb-0cdc32a30916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:39.453821 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cdde1c-1d3d-4dc1-94fb-0cdc32a30916 admin admin] PATH_INFO: `/v3/users/d0507f846e0a4f26bbeda8524c7fafe0` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:39.457314 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cdde1c-1d3d-4dc1-94fb-0cdc32a30916 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0507f846e0a4f26bbeda8524c7fafe0)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:39.461361 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: WARNING py.warnings [None req-a0cdde1c-1d3d-4dc1-94fb-0cdc32a30916 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 27 09:36:39.461361 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: warnings.warn(msg) Oct 27 09:36:39.461361 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: Oct 27 09:36:39.468467 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cdde1c-1d3d-4dc1-94fb-0cdc32a30916 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:39.473601 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812c47a2-1da1-44e8-b8f0-30c2afb02d2b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:39.482302 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812c47a2-1da1-44e8-b8f0-30c2afb02d2b service placement] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:39.520104 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 678/1336] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:36:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 09:36:39.631449 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a0cdde1c-1d3d-4dc1-94fb-0cdc32a30916 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0507f846e0a4f26bbeda8524c7fafe0'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:39.674905 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a0cdde1c-1d3d-4dc1-94fb-0cdc32a30916 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0507f846e0a4f26bbeda8524c7fafe0'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:39.680943 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c47a2-1da1-44e8-b8f0-30c2afb02d2b service placement] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:39.707269 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c47a2-1da1-44e8-b8f0-30c2afb02d2b service placement] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:39.721986 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a0cdde1c-1d3d-4dc1-94fb-0cdc32a30916 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0507f846e0a4f26bbeda8524c7fafe0'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:39.728893 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b12150-36dc-40a9-ae12-01a658806e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:39.730045 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b12150-36dc-40a9-ae12-01a658806e60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:39.730569 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b12150-36dc-40a9-ae12-01a658806e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:39.733474 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b12150-36dc-40a9-ae12-01a658806e60 admin admin] PATH_INFO: `/v3/users/4ccf46600901461e880a41188a92696f` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:39.735154 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b12150-36dc-40a9-ae12-01a658806e60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ccf46600901461e880a41188a92696f)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:39.737440 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b12150-36dc-40a9-ae12-01a658806e60 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:39.758043 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a0cdde1c-1d3d-4dc1-94fb-0cdc32a30916 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0507f846e0a4f26bbeda8524c7fafe0'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:39.759092 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a0cdde1c-1d3d-4dc1-94fb-0cdc32a30916 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0507f846e0a4f26bbeda8524c7fafe0'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:39.774364 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 659/1337] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:36:39 2022] DELETE /identity/v3/users/d0507f846e0a4f26bbeda8524c7fafe0 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:36:39.780981 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349703b7-e587-46fa-99e4-366e140336fb None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:39.781859 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349703b7-e587-46fa-99e4-366e140336fb None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:39.782100 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349703b7-e587-46fa-99e4-366e140336fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:39.867319 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-349703b7-e587-46fa-99e4-366e140336fb None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:39.885597 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-349703b7-e587-46fa-99e4-366e140336fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:39.954205 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-79b12150-36dc-40a9-ae12-01a658806e60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ccf46600901461e880a41188a92696f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:39.990714 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-79b12150-36dc-40a9-ae12-01a658806e60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ccf46600901461e880a41188a92696f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:40.022448 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-79b12150-36dc-40a9-ae12-01a658806e60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ccf46600901461e880a41188a92696f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:40.047757 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-79b12150-36dc-40a9-ae12-01a658806e60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ccf46600901461e880a41188a92696f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:40.048641 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-79b12150-36dc-40a9-ae12-01a658806e60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ccf46600901461e880a41188a92696f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:40.068824 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:40.075116 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 679/1338] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:36:39 2022] DELETE /identity/v3/users/4ccf46600901461e880a41188a92696f => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:36:40.076877 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc87e5-9a8f-42d5-9599-854d59491e29 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:40.077436 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc87e5-9a8f-42d5-9599-854d59491e29 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:40.078845 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc87e5-9a8f-42d5-9599-854d59491e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:40.103675 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 660/1339] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:40.162131 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349703b7-e587-46fa-99e4-366e140336fb None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:40.165115 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-349703b7-e587-46fa-99e4-366e140336fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:40.174273 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-6efc87e5-9a8f-42d5-9599-854d59491e29 None None] MFA Rules not processed for user `286df4c587d7408d988c4739f4ff923a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:40.179426 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6efc87e5-9a8f-42d5-9599-854d59491e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:40.210531 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:40.383772 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349703b7-e587-46fa-99e4-366e140336fb None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:40.454962 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 680/1340] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:40.475602 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc87e5-9a8f-42d5-9599-854d59491e29 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:40.477124 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6efc87e5-9a8f-42d5-9599-854d59491e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:40.581649 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ba4ccc-435d-4305-b056-d89c05ba580b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:40.583603 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba4ccc-435d-4305-b056-d89c05ba580b service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:40.584071 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba4ccc-435d-4305-b056-d89c05ba580b service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:40.584731 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba4ccc-435d-4305-b056-d89c05ba580b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:40.660392 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc87e5-9a8f-42d5-9599-854d59491e29 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:40.673318 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d289d3e8-f2da-4fc1-9093-a6d067502e33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe4a6e4b6184826b30ac4fc13ad9b1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:40.673992 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289d3e8-f2da-4fc1-9093-a6d067502e33 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:40.674302 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289d3e8-f2da-4fc1-9093-a6d067502e33 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:40.674556 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289d3e8-f2da-4fc1-9093-a6d067502e33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:40.694571 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d289d3e8-f2da-4fc1-9093-a6d067502e33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:40.697163 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d289d3e8-f2da-4fc1-9093-a6d067502e33 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:40.715579 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 681/1341] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:36:40 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:36:40.776897 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ba4ccc-435d-4305-b056-d89c05ba580b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:40.784814 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ba4ccc-435d-4305-b056-d89c05ba580b service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:40.808117 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 661/1342] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:36:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 646 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 09:36:41.865969 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d289d3e8-f2da-4fc1-9093-a6d067502e33 service nova] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:41.867481 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d289d3e8-f2da-4fc1-9093-a6d067502e33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:41.910742 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d289d3e8-f2da-4fc1-9093-a6d067502e33 service nova] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:41.930557 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b3323-c62c-450a-8164-3ba3410d6257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:41.934460 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b3323-c62c-450a-8164-3ba3410d6257 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:41.934712 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b3323-c62c-450a-8164-3ba3410d6257 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:41.934997 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b3323-c62c-450a-8164-3ba3410d6257 admin admin] PATH_INFO: `/v3/projects/81430525775d465bb6b8268d1d587def` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:41.942443 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19b3323-c62c-450a-8164-3ba3410d6257 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81430525775d465bb6b8268d1d587def)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:41.948264 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: WARNING py.warnings [None req-a19b3323-c62c-450a-8164-3ba3410d6257 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 27 09:36:41.948264 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: warnings.warn(msg) Oct 27 09:36:41.948264 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: Oct 27 09:36:41.954352 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19b3323-c62c-450a-8164-3ba3410d6257 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:42.074385 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a19b3323-c62c-450a-8164-3ba3410d6257 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81430525775d465bb6b8268d1d587def', 'request_id': 'req-a19b3323-c62c-450a-8164-3ba3410d6257'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:42.084486 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a19b3323-c62c-450a-8164-3ba3410d6257 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81430525775d465bb6b8268d1d587def', 'request_id': 'req-a19b3323-c62c-450a-8164-3ba3410d6257'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:42.104478 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a19b3323-c62c-450a-8164-3ba3410d6257 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81430525775d465bb6b8268d1d587def', 'request_id': 'req-a19b3323-c62c-450a-8164-3ba3410d6257'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:42.163167 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a19b3323-c62c-450a-8164-3ba3410d6257 admin admin] The token cache is being invalidate because project 81430525775d465bb6b8268d1d587def was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:36:42.163756 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a19b3323-c62c-450a-8164-3ba3410d6257 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:42.164869 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a19b3323-c62c-450a-8164-3ba3410d6257 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:42.168199 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 682/1343] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:36:41 2022] DELETE /identity/v3/projects/81430525775d465bb6b8268d1d587def => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:36:42.172899 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ba4ccc-435d-4305-b056-d89c05ba580b service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:42.187414 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ba4ccc-435d-4305-b056-d89c05ba580b service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:42.342760 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:42.355387 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de4691a-6fed-45bd-ad97-8e387d027469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:42.356230 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de4691a-6fed-45bd-ad97-8e387d027469 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:42.356713 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de4691a-6fed-45bd-ad97-8e387d027469 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:42.357131 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de4691a-6fed-45bd-ad97-8e387d027469 admin admin] PATH_INFO: `/v3/users/c038d9499edf46e59c1ba7b9a00758e9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:42.358798 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de4691a-6fed-45bd-ad97-8e387d027469 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c038d9499edf46e59c1ba7b9a00758e9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:42.365566 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de4691a-6fed-45bd-ad97-8e387d027469 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:42.455806 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3c1bb-2ae8-4b0b-877f-a7aca46fc0fc None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:42.456878 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3c1bb-2ae8-4b0b-877f-a7aca46fc0fc None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:42.457339 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3c1bb-2ae8-4b0b-877f-a7aca46fc0fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:42.510370 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6de4691a-6fed-45bd-ad97-8e387d027469 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c038d9499edf46e59c1ba7b9a00758e9'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:42.517074 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-3cb3c1bb-2ae8-4b0b-877f-a7aca46fc0fc None None] MFA Rules not processed for user `2dea51c29cbd46e8bd1454e10eaac012`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:42.522951 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3cb3c1bb-2ae8-4b0b-877f-a7aca46fc0fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:42.528152 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6de4691a-6fed-45bd-ad97-8e387d027469 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c038d9499edf46e59c1ba7b9a00758e9'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:42.541534 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:42.586044 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6de4691a-6fed-45bd-ad97-8e387d027469 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c038d9499edf46e59c1ba7b9a00758e9'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:42.610237 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6de4691a-6fed-45bd-ad97-8e387d027469 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c038d9499edf46e59c1ba7b9a00758e9'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:42.610934 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6de4691a-6fed-45bd-ad97-8e387d027469 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c038d9499edf46e59c1ba7b9a00758e9'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:42.621713 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 662/1344] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:36:42 2022] DELETE /identity/v3/users/c038d9499edf46e59c1ba7b9a00758e9 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:36:42.664470 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 683/1345] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:42 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:42.676716 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de4691a-6fed-45bd-ad97-8e387d027469 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:42.678712 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6de4691a-6fed-45bd-ad97-8e387d027469 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:42.858165 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de4691a-6fed-45bd-ad97-8e387d027469 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:42.883608 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222623fe-8ac9-48bf-bfe1-b7f918269c02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe4a6e4b6184826b30ac4fc13ad9b1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:42.884445 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222623fe-8ac9-48bf-bfe1-b7f918269c02 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:42.884950 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222623fe-8ac9-48bf-bfe1-b7f918269c02 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:42.885330 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222623fe-8ac9-48bf-bfe1-b7f918269c02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:42.986838 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb3c1bb-2ae8-4b0b-877f-a7aca46fc0fc None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:42.989566 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3cb3c1bb-2ae8-4b0b-877f-a7aca46fc0fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:43.028707 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb3c1bb-2ae8-4b0b-877f-a7aca46fc0fc None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:43.069824 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222623fe-8ac9-48bf-bfe1-b7f918269c02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:43.071592 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222623fe-8ac9-48bf-bfe1-b7f918269c02 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:43.093145 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 663/1346] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:36:42 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:36:43.199761 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2ea9cb-1a8b-4bcc-bd4e-5c292a64e547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:43.199761 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ea9cb-1a8b-4bcc-bd4e-5c292a64e547 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:43.199761 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ea9cb-1a8b-4bcc-bd4e-5c292a64e547 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:43.199761 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ea9cb-1a8b-4bcc-bd4e-5c292a64e547 admin admin] PATH_INFO: `/v3/projects/4e6f8f816ed2443791470c0402ed87df` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:43.204644 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2ea9cb-1a8b-4bcc-bd4e-5c292a64e547 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e6f8f816ed2443791470c0402ed87df)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:43.212320 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2ea9cb-1a8b-4bcc-bd4e-5c292a64e547 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:43.373107 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-df2ea9cb-1a8b-4bcc-bd4e-5c292a64e547 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e6f8f816ed2443791470c0402ed87df', 'request_id': 'req-df2ea9cb-1a8b-4bcc-bd4e-5c292a64e547'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:43.402240 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-df2ea9cb-1a8b-4bcc-bd4e-5c292a64e547 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e6f8f816ed2443791470c0402ed87df', 'request_id': 'req-df2ea9cb-1a8b-4bcc-bd4e-5c292a64e547'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:43.431549 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-df2ea9cb-1a8b-4bcc-bd4e-5c292a64e547 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e6f8f816ed2443791470c0402ed87df', 'request_id': 'req-df2ea9cb-1a8b-4bcc-bd4e-5c292a64e547'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:43.467898 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-df2ea9cb-1a8b-4bcc-bd4e-5c292a64e547 admin admin] The token cache is being invalidate because project 4e6f8f816ed2443791470c0402ed87df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:36:43.468387 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-df2ea9cb-1a8b-4bcc-bd4e-5c292a64e547 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:43.469324 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-df2ea9cb-1a8b-4bcc-bd4e-5c292a64e547 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:43.477169 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 684/1347] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:36:42 2022] DELETE /identity/v3/projects/4e6f8f816ed2443791470c0402ed87df => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:36:43.478238 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c16d148-490d-4ae5-b2af-bb9bfd8c6e97 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:43.479069 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c16d148-490d-4ae5-b2af-bb9bfd8c6e97 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:43.479496 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c16d148-490d-4ae5-b2af-bb9bfd8c6e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:43.547391 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-4c16d148-490d-4ae5-b2af-bb9bfd8c6e97 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:43.549975 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4c16d148-490d-4ae5-b2af-bb9bfd8c6e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:43.596350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:43.824507 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 664/1348] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:43.833840 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b621c0e-5161-492f-aefa-febcf6994f33 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:43.833840 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b621c0e-5161-492f-aefa-febcf6994f33 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:43.833840 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b621c0e-5161-492f-aefa-febcf6994f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:43.887985 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-0b621c0e-5161-492f-aefa-febcf6994f33 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:43.888862 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0b621c0e-5161-492f-aefa-febcf6994f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:43.918898 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 685/1349] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:43.920212 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c16d148-490d-4ae5-b2af-bb9bfd8c6e97 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:43.935521 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c16d148-490d-4ae5-b2af-bb9bfd8c6e97 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:43.948713 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5712e5-4426-4fbb-a0d6-c07a5af41b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:43.949721 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5712e5-4426-4fbb-a0d6-c07a5af41b38 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:43.950006 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5712e5-4426-4fbb-a0d6-c07a5af41b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:43.951657 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5712e5-4426-4fbb-a0d6-c07a5af41b38 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:43.953349 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5712e5-4426-4fbb-a0d6-c07a5af41b38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:36:43.953854 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5712e5-4426-4fbb-a0d6-c07a5af41b38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:43.959195 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5712e5-4426-4fbb-a0d6-c07a5af41b38 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:43.993496 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 665/1350] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:36:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:36:43.995157 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b621c0e-5161-492f-aefa-febcf6994f33 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:44.009825 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b621c0e-5161-492f-aefa-febcf6994f33 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:44.037662 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1566b8-e6ac-44c1-ac8a-ff38491dad14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:44.038331 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1566b8-e6ac-44c1-ac8a-ff38491dad14 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:44.038657 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1566b8-e6ac-44c1-ac8a-ff38491dad14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:44.042991 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1566b8-e6ac-44c1-ac8a-ff38491dad14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:44.044000 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1566b8-e6ac-44c1-ac8a-ff38491dad14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:44.052594 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5712e5-4426-4fbb-a0d6-c07a5af41b38 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:44.054640 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ad5712e5-4426-4fbb-a0d6-c07a5af41b38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:44.066584 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1566b8-e6ac-44c1-ac8a-ff38491dad14 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:44.099322 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5712e5-4426-4fbb-a0d6-c07a5af41b38 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:44.124584 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a59f732-5aa0-49c9-882e-c631ac0d83e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:44.125353 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59f732-5aa0-49c9-882e-c631ac0d83e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:44.125638 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59f732-5aa0-49c9-882e-c631ac0d83e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:44.125915 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59f732-5aa0-49c9-882e-c631ac0d83e8 admin admin] PATH_INFO: `/v3/projects/19c0e11ecca94dcbb9eaace33e413f9f` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:44.136393 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a59f732-5aa0-49c9-882e-c631ac0d83e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19c0e11ecca94dcbb9eaace33e413f9f)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:44.149468 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a59f732-5aa0-49c9-882e-c631ac0d83e8 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:44.209892 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 686/1351] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:36:43 2022] POST /identity/v3/projects => generated 371 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:36:44.214206 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1566b8-e6ac-44c1-ac8a-ff38491dad14 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:44.244040 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:44.292655 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6a59f732-5aa0-49c9-882e-c631ac0d83e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19c0e11ecca94dcbb9eaace33e413f9f', 'request_id': 'req-6a59f732-5aa0-49c9-882e-c631ac0d83e8'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:44.304284 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6a59f732-5aa0-49c9-882e-c631ac0d83e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19c0e11ecca94dcbb9eaace33e413f9f', 'request_id': 'req-6a59f732-5aa0-49c9-882e-c631ac0d83e8'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:44.321925 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6a59f732-5aa0-49c9-882e-c631ac0d83e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19c0e11ecca94dcbb9eaace33e413f9f', 'request_id': 'req-6a59f732-5aa0-49c9-882e-c631ac0d83e8'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:44.362537 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6a59f732-5aa0-49c9-882e-c631ac0d83e8 admin admin] The token cache is being invalidate because project 19c0e11ecca94dcbb9eaace33e413f9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:36:44.364234 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6a59f732-5aa0-49c9-882e-c631ac0d83e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:44.365994 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6a59f732-5aa0-49c9-882e-c631ac0d83e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:44.369746 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 666/1352] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:36:44 2022] DELETE /identity/v3/projects/19c0e11ecca94dcbb9eaace33e413f9f => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:36:44.375425 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c08b7-5090-4141-af34-e8cd7e5b2a81 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:44.375885 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c08b7-5090-4141-af34-e8cd7e5b2a81 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:44.376161 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c08b7-5090-4141-af34-e8cd7e5b2a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:44.404644 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1566b8-e6ac-44c1-ac8a-ff38491dad14 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:44.416245 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-d53c08b7-5090-4141-af34-e8cd7e5b2a81 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:44.418148 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d53c08b7-5090-4141-af34-e8cd7e5b2a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:44.517552 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416e878f-eaea-45ce-b044-ce7d8c6fa1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:44.518611 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e878f-eaea-45ce-b044-ce7d8c6fa1c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:44.518980 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e878f-eaea-45ce-b044-ce7d8c6fa1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:44.519460 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e878f-eaea-45ce-b044-ce7d8c6fa1c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:44.520366 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416e878f-eaea-45ce-b044-ce7d8c6fa1c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:44.523532 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416e878f-eaea-45ce-b044-ce7d8c6fa1c1 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:44.550735 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 667/1353] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:36:44.561935 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f7663-e142-4324-8032-87b43f35c0e4 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:44.563366 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f7663-e142-4324-8032-87b43f35c0e4 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:44.564076 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f7663-e142-4324-8032-87b43f35c0e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:44.569365 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 687/1354] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:36:44 2022] POST /identity/v3/users => generated 286 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:36:44.571970 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416e878f-eaea-45ce-b044-ce7d8c6fa1c1 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:44.578918 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-416e878f-eaea-45ce-b044-ce7d8c6fa1c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:44.643321 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416e878f-eaea-45ce-b044-ce7d8c6fa1c1 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:44.645921 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-9b5f7663-e142-4324-8032-87b43f35c0e4 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:44.656395 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9b5f7663-e142-4324-8032-87b43f35c0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:44.679643 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef82839-32e0-4ee3-aff6-b0041b1c7820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:44.682526 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef82839-32e0-4ee3-aff6-b0041b1c7820 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:44.682526 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef82839-32e0-4ee3-aff6-b0041b1c7820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:44.685037 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef82839-32e0-4ee3-aff6-b0041b1c7820 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:44.688632 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef82839-32e0-4ee3-aff6-b0041b1c7820 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:44.695454 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef82839-32e0-4ee3-aff6-b0041b1c7820 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:44.700322 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 668/1355] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:44.703885 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5d2c7-141d-46bd-ae41-f7591c7af25a None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:44.707596 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5d2c7-141d-46bd-ae41-f7591c7af25a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:44.708417 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5d2c7-141d-46bd-ae41-f7591c7af25a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:44.735312 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 688/1356] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:36:44 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:36:44.736395 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef82839-32e0-4ee3-aff6-b0041b1c7820 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:44.768272 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef82839-32e0-4ee3-aff6-b0041b1c7820 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:44.782492 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e97b1f-9b9a-431f-b736-f7aa647883b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:44.783364 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e97b1f-9b9a-431f-b736-f7aa647883b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:44.783571 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-41c5d2c7-141d-46bd-ae41-f7591c7af25a None None] MFA Rules not processed for user `20c65ed67cd8403489949c4dea572d01`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:44.784065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e97b1f-9b9a-431f-b736-f7aa647883b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:44.785498 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-41c5d2c7-141d-46bd-ae41-f7591c7af25a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:44.792968 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e97b1f-9b9a-431f-b736-f7aa647883b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:44.792968 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e97b1f-9b9a-431f-b736-f7aa647883b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:36:44.792968 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e97b1f-9b9a-431f-b736-f7aa647883b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:44.792968 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e97b1f-9b9a-431f-b736-f7aa647883b9 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:44.805353 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:44.821573 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 689/1357] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:36:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:36:44.821573 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e97b1f-9b9a-431f-b736-f7aa647883b9 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:44.849315 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e97b1f-9b9a-431f-b736-f7aa647883b9 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:44.860068 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2532f12-85d0-41af-992d-5937f7f089f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:44.860725 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2532f12-85d0-41af-992d-5937f7f089f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:44.861005 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2532f12-85d0-41af-992d-5937f7f089f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:44.861275 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2532f12-85d0-41af-992d-5937f7f089f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:44.862024 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2532f12-85d0-41af-992d-5937f7f089f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:44.867642 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2532f12-85d0-41af-992d-5937f7f089f1 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:44.890890 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 690/1358] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:36:44 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:36:44.890890 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2532f12-85d0-41af-992d-5937f7f089f1 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:44.911591 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2532f12-85d0-41af-992d-5937f7f089f1 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:44.923698 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841de669-8e9b-4aab-b9f9-1681df215bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:44.926498 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841de669-8e9b-4aab-b9f9-1681df215bb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:44.927789 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841de669-8e9b-4aab-b9f9-1681df215bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:44.930018 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841de669-8e9b-4aab-b9f9-1681df215bb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:44.930985 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841de669-8e9b-4aab-b9f9-1681df215bb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:44.935225 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841de669-8e9b-4aab-b9f9-1681df215bb8 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:44.946414 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 669/1359] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:44 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:44.948088 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c5d2c7-141d-46bd-ae41-f7591c7af25a None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:44.974253 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c5d2c7-141d-46bd-ae41-f7591c7af25a None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:44.980191 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae56ed-ecc7-427d-a029-0263dab13054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:44.981112 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae56ed-ecc7-427d-a029-0263dab13054 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:44.981492 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae56ed-ecc7-427d-a029-0263dab13054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:44.981868 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae56ed-ecc7-427d-a029-0263dab13054 admin admin] PATH_INFO: `/v3/projects/c393eb39d1ca42aa879756bb8d7ec4e3/users/7056c7a20375483f89d5a535188a4a7b/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:45.005669 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ae56ed-ecc7-427d-a029-0263dab13054 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c393eb39d1ca42aa879756bb8d7ec4e3, user_id=7056c7a20375483f89d5a535188a4a7b, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:45.009906 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ae56ed-ecc7-427d-a029-0263dab13054 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:45.033159 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 691/1360] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:36:44 2022] POST /identity/v3/projects => generated 385 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:36:45.059615 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85977c9-d3fb-4082-a9bd-51018d0314bd None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:45.059888 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85977c9-d3fb-4082-a9bd-51018d0314bd None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:45.060471 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85977c9-d3fb-4082-a9bd-51018d0314bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:45.071915 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 670/1361] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:36:44 2022] PUT /identity/v3/projects/c393eb39d1ca42aa879756bb8d7ec4e3/users/7056c7a20375483f89d5a535188a4a7b/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:36:45.074207 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae56ed-ecc7-427d-a029-0263dab13054 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:45.111826 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-a85977c9-d3fb-4082-a9bd-51018d0314bd None None] MFA Rules not processed for user `20c65ed67cd8403489949c4dea572d01`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:45.114767 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a85977c9-d3fb-4082-a9bd-51018d0314bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:45.239628 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:45.275182 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae56ed-ecc7-427d-a029-0263dab13054 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:45.287629 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 692/1362] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:45 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:45.288653 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e378c83f-820b-43b9-ae02-0825fb955dda None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:45.288943 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e378c83f-820b-43b9-ae02-0825fb955dda None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:45.290709 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e378c83f-820b-43b9-ae02-0825fb955dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:45.370865 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-e378c83f-820b-43b9-ae02-0825fb955dda None None] MFA Rules not processed for user `7056c7a20375483f89d5a535188a4a7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:45.372718 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e378c83f-820b-43b9-ae02-0825fb955dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:45.433565 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab808c1-6890-4416-83f9-ca800d996ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:45.434282 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab808c1-6890-4416-83f9-ca800d996ae7 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:45.434564 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab808c1-6890-4416-83f9-ca800d996ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:45.434842 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab808c1-6890-4416-83f9-ca800d996ae7 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:45.435842 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab808c1-6890-4416-83f9-ca800d996ae7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:45.438484 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab808c1-6890-4416-83f9-ca800d996ae7 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:45.522247 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 671/1363] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:36:45 2022] POST /identity/v3/users => generated 293 bytes in 449 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:36:45.530357 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab808c1-6890-4416-83f9-ca800d996ae7 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:45.540038 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 693/1364] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:45.541696 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb642207-5680-40a7-93f7-cc32e92ec539 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:45.542130 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb642207-5680-40a7-93f7-cc32e92ec539 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:45.542536 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb642207-5680-40a7-93f7-cc32e92ec539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:45.544231 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3ab808c1-6890-4416-83f9-ca800d996ae7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:45.611337 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:45.627963 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-eb642207-5680-40a7-93f7-cc32e92ec539 None None] MFA Rules not processed for user `6b8fa2a89eee41d09977d7c85c49707b`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:45.635486 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-eb642207-5680-40a7-93f7-cc32e92ec539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:45.791760 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab808c1-6890-4416-83f9-ca800d996ae7 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:45.806641 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136f5af-8015-45c0-b878-5ccd1db8bde8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe4a6e4b6184826b30ac4fc13ad9b1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:45.807456 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136f5af-8015-45c0-b878-5ccd1db8bde8 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:45.807832 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136f5af-8015-45c0-b878-5ccd1db8bde8 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:45.808196 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136f5af-8015-45c0-b878-5ccd1db8bde8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:45.823814 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0136f5af-8015-45c0-b878-5ccd1db8bde8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:45.827489 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0136f5af-8015-45c0-b878-5ccd1db8bde8 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:45.832064 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 694/1365] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:45.833618 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb642207-5680-40a7-93f7-cc32e92ec539 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:45.850365 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 672/1366] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:36:45 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:36:45.851330 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136f5af-8015-45c0-b878-5ccd1db8bde8 service nova] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:45.853637 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0136f5af-8015-45c0-b878-5ccd1db8bde8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:45.875138 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb642207-5680-40a7-93f7-cc32e92ec539 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:45.889272 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136f5af-8015-45c0-b878-5ccd1db8bde8 service nova] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:45.902382 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b75f6d5-0ecc-4e42-a266-9ad782980188 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe4a6e4b6184826b30ac4fc13ad9b1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:45.903264 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b75f6d5-0ecc-4e42-a266-9ad782980188 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:45.903650 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b75f6d5-0ecc-4e42-a266-9ad782980188 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:45.904025 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b75f6d5-0ecc-4e42-a266-9ad782980188 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:45.915590 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5243abe0-bd31-4662-a1da-df7bf17e4b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:45.919259 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5243abe0-bd31-4662-a1da-df7bf17e4b6c admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:45.922172 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5243abe0-bd31-4662-a1da-df7bf17e4b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:45.923461 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5243abe0-bd31-4662-a1da-df7bf17e4b6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:45.924272 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5243abe0-bd31-4662-a1da-df7bf17e4b6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:45.937136 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5243abe0-bd31-4662-a1da-df7bf17e4b6c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:45.941724 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b75f6d5-0ecc-4e42-a266-9ad782980188 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:45.953801 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b75f6d5-0ecc-4e42-a266-9ad782980188 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:45.971948 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 695/1367] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:36:45.976884 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659171ac-33e6-4e04-9705-ee0d80e179b0 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:45.977288 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659171ac-33e6-4e04-9705-ee0d80e179b0 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:45.979365 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659171ac-33e6-4e04-9705-ee0d80e179b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:45.994213 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 673/1368] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:36:45 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:36:46.001302 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b75f6d5-0ecc-4e42-a266-9ad782980188 service nova] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:46.039770 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-659171ac-33e6-4e04-9705-ee0d80e179b0 None None] MFA Rules not processed for user `6b8fa2a89eee41d09977d7c85c49707b`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:46.045488 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-659171ac-33e6-4e04-9705-ee0d80e179b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:46.046300 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b75f6d5-0ecc-4e42-a266-9ad782980188 service nova] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:46.061201 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 696/1369] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:46.061959 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a091c07-f23c-4303-a2ad-2fb6461247da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:46.062819 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a091c07-f23c-4303-a2ad-2fb6461247da admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:46.063607 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a091c07-f23c-4303-a2ad-2fb6461247da admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:46.064114 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a091c07-f23c-4303-a2ad-2fb6461247da admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:46.065193 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a091c07-f23c-4303-a2ad-2fb6461247da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:46.075321 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a091c07-f23c-4303-a2ad-2fb6461247da admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:46.103355 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 674/1370] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:36:46.107285 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659171ac-33e6-4e04-9705-ee0d80e179b0 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:46.109709 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a091c07-f23c-4303-a2ad-2fb6461247da admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:46.138343 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659171ac-33e6-4e04-9705-ee0d80e179b0 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:46.147358 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b240e5-3081-4330-9758-7fd5e28d54bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe4a6e4b6184826b30ac4fc13ad9b1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:46.148104 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b240e5-3081-4330-9758-7fd5e28d54bf service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:46.148485 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b240e5-3081-4330-9758-7fd5e28d54bf service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:46.149041 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b240e5-3081-4330-9758-7fd5e28d54bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:46.149233 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a091c07-f23c-4303-a2ad-2fb6461247da admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:46.164424 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79272836-ff78-4850-90b5-c85665c075a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:46.168206 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b240e5-3081-4330-9758-7fd5e28d54bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:46.169042 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79272836-ff78-4850-90b5-c85665c075a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:46.169327 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79272836-ff78-4850-90b5-c85665c075a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:46.169603 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79272836-ff78-4850-90b5-c85665c075a1 admin admin] PATH_INFO: `/v3/projects/836ce9e6021a4fcd87c892a094d5a1a4/users/bff29d778f5d48548aad0ef6bd47eea3/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:46.172247 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b240e5-3081-4330-9758-7fd5e28d54bf service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:46.207304 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 697/1371] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:36:46 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 09:36:46.215341 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79272836-ff78-4850-90b5-c85665c075a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=836ce9e6021a4fcd87c892a094d5a1a4, user_id=bff29d778f5d48548aad0ef6bd47eea3, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:46.222091 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79272836-ff78-4850-90b5-c85665c075a1 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:46.266894 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 675/1372] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:36:46 2022] PUT /identity/v3/projects/836ce9e6021a4fcd87c892a094d5a1a4/users/bff29d778f5d48548aad0ef6bd47eea3/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:36:46.268872 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554be79e-8395-4260-b2c5-dbefda96e797 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:46.269888 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554be79e-8395-4260-b2c5-dbefda96e797 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:46.269888 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554be79e-8395-4260-b2c5-dbefda96e797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:46.346283 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-554be79e-8395-4260-b2c5-dbefda96e797 None None] MFA Rules not processed for user `bff29d778f5d48548aad0ef6bd47eea3`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:46.348717 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-554be79e-8395-4260-b2c5-dbefda96e797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:46.421309 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:46.488071 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 698/1373] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:46.491132 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79272836-ff78-4850-90b5-c85665c075a1 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:46.547172 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff493d-ee57-472e-886b-edca8d2eac71 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:46.547503 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff493d-ee57-472e-886b-edca8d2eac71 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:46.547914 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff493d-ee57-472e-886b-edca8d2eac71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:46.596958 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-ddff493d-ee57-472e-886b-edca8d2eac71 None None] MFA Rules not processed for user `286df4c587d7408d988c4739f4ff923a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:46.599892 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ddff493d-ee57-472e-886b-edca8d2eac71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:46.654013 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79272836-ff78-4850-90b5-c85665c075a1 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:46.667204 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d1bd70-eded-4af2-9d29-a5b3b30f2e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:46.667204 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d1bd70-eded-4af2-9d29-a5b3b30f2e03 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:46.667204 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d1bd70-eded-4af2-9d29-a5b3b30f2e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:46.667204 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d1bd70-eded-4af2-9d29-a5b3b30f2e03 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:46.669714 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d1bd70-eded-4af2-9d29-a5b3b30f2e03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:46.679532 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d1bd70-eded-4af2-9d29-a5b3b30f2e03 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:46.755153 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 676/1374] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:36:46 2022] POST /identity/v3/projects => generated 387 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:36:46.757904 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d1bd70-eded-4af2-9d29-a5b3b30f2e03 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:46.770939 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 699/1375] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:36:46.876854 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:46.925588 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d1bd70-eded-4af2-9d29-a5b3b30f2e03 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:46.945340 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a63b15-31dd-4c3e-8244-393b3b3d8886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:46.945340 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a63b15-31dd-4c3e-8244-393b3b3d8886 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:46.945340 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a63b15-31dd-4c3e-8244-393b3b3d8886 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:46.945726 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a63b15-31dd-4c3e-8244-393b3b3d8886 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:46.946236 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a63b15-31dd-4c3e-8244-393b3b3d8886 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:46.948214 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a63b15-31dd-4c3e-8244-393b3b3d8886 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:47.009286 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 677/1376] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:36:46 2022] POST /identity/v3/users => generated 300 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:36:47.010487 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddff493d-ee57-472e-886b-edca8d2eac71 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:47.031212 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddff493d-ee57-472e-886b-edca8d2eac71 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:47.051023 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac026f-1bc4-45e2-9111-989faa398d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:47.070816 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac026f-1bc4-45e2-9111-989faa398d14 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:47.071423 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac026f-1bc4-45e2-9111-989faa398d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:47.072998 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac026f-1bc4-45e2-9111-989faa398d14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:47.076116 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ac026f-1bc4-45e2-9111-989faa398d14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:47.078493 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ac026f-1bc4-45e2-9111-989faa398d14 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:47.131692 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 700/1377] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:36:47 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 09:36:47.136728 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a63b15-31dd-4c3e-8244-393b3b3d8886 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:47.149520 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a63b15-31dd-4c3e-8244-393b3b3d8886 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:47.162679 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a13b2-35bd-45ea-9e6f-708da62e3c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:47.162679 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a13b2-35bd-45ea-9e6f-708da62e3c61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:47.162679 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a13b2-35bd-45ea-9e6f-708da62e3c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:47.163064 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a13b2-35bd-45ea-9e6f-708da62e3c61 admin admin] PATH_INFO: `/v3/projects/f2946c1d27dc41378f2453e89b11fbd5/users/43175f3f693740cb8a0fb8f43d91c8f9/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:47.197254 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141a13b2-35bd-45ea-9e6f-708da62e3c61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2946c1d27dc41378f2453e89b11fbd5, user_id=43175f3f693740cb8a0fb8f43d91c8f9, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:47.227109 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141a13b2-35bd-45ea-9e6f-708da62e3c61 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:47.253784 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 678/1378] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:36:47 2022] PUT /identity/v3/projects/f2946c1d27dc41378f2453e89b11fbd5/users/43175f3f693740cb8a0fb8f43d91c8f9/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:36:47.257549 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac026f-1bc4-45e2-9111-989faa398d14 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:47.271706 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac026f-1bc4-45e2-9111-989faa398d14 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:47.441058 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e986dc5-d361-4c26-8fa3-af0daeaf4cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:47.441923 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e986dc5-d361-4c26-8fa3-af0daeaf4cc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:47.442339 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e986dc5-d361-4c26-8fa3-af0daeaf4cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:47.442756 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e986dc5-d361-4c26-8fa3-af0daeaf4cc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:47.443721 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e986dc5-d361-4c26-8fa3-af0daeaf4cc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:47.447212 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e986dc5-d361-4c26-8fa3-af0daeaf4cc3 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:47.478652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 701/1379] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:36:47 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:36:47.480827 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a13b2-35bd-45ea-9e6f-708da62e3c61 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:47.497072 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a13b2-35bd-45ea-9e6f-708da62e3c61 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:47.513176 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d49316c-e536-47aa-8bab-b517050e5ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:47.514638 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d49316c-e536-47aa-8bab-b517050e5ea0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:47.515983 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d49316c-e536-47aa-8bab-b517050e5ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:47.517718 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d49316c-e536-47aa-8bab-b517050e5ea0 admin admin] PATH_INFO: `/v3/projects/f2946c1d27dc41378f2453e89b11fbd5/users/43175f3f693740cb8a0fb8f43d91c8f9/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:47.521814 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d49316c-e536-47aa-8bab-b517050e5ea0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2946c1d27dc41378f2453e89b11fbd5, user_id=43175f3f693740cb8a0fb8f43d91c8f9, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:47.530126 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d49316c-e536-47aa-8bab-b517050e5ea0 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:47.559704 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 679/1380] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:36:47 2022] PUT /identity/v3/projects/f2946c1d27dc41378f2453e89b11fbd5/users/43175f3f693740cb8a0fb8f43d91c8f9/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:36:47.560318 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511935d9-b03c-4996-8795-3208f0370de8 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:47.560658 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511935d9-b03c-4996-8795-3208f0370de8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:47.560919 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511935d9-b03c-4996-8795-3208f0370de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:47.583323 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f936d-dc4f-4247-9cd7-2ba0d10f175a None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:47.583827 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f936d-dc4f-4247-9cd7-2ba0d10f175a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:47.584339 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f936d-dc4f-4247-9cd7-2ba0d10f175a None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:47.586467 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 680/1381] 10.222.0.70 () {54 vars in 912 bytes} [Thu Oct 27 09:36:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 27 09:36:47.592147 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a82fbb-10f1-475a-bf56-8c72a395d80d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:47.593058 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a82fbb-10f1-475a-bf56-8c72a395d80d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:47.593619 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a82fbb-10f1-475a-bf56-8c72a395d80d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:47.632066 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-511935d9-b03c-4996-8795-3208f0370de8 None None] MFA Rules not processed for user `43175f3f693740cb8a0fb8f43d91c8f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:47.633333 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-511935d9-b03c-4996-8795-3208f0370de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:47.655683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-85a82fbb-10f1-475a-bf56-8c72a395d80d None None] MFA Rules not processed for user `45d8920d9a7940abb112dbbb3d7f6e65`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:47.657821 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-85a82fbb-10f1-475a-bf56-8c72a395d80d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:47.785677 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:47.806764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 681/1382] 10.222.0.70 () {58 vars in 1009 bytes} [Thu Oct 27 09:36:47 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:36:47.872899 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 702/1383] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:47 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:36:47.875382 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a82fbb-10f1-475a-bf56-8c72a395d80d None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:47.883407 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-85a82fbb-10f1-475a-bf56-8c72a395d80d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:47.883773 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511935d9-b03c-4996-8795-3208f0370de8 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:47.885579 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-511935d9-b03c-4996-8795-3208f0370de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:47.900484 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a82fbb-10f1-475a-bf56-8c72a395d80d None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:47.922620 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d9f404-60f9-4844-af64-f33c6aefb9e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:47.924281 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9f404-60f9-4844-af64-f33c6aefb9e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:47.924779 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9f404-60f9-4844-af64-f33c6aefb9e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:47.925058 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9f404-60f9-4844-af64-f33c6aefb9e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:47.950798 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d9f404-60f9-4844-af64-f33c6aefb9e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:47.952223 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d9f404-60f9-4844-af64-f33c6aefb9e2 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:47.981186 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 682/1384] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:36:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:36:47.984803 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e71563-4730-475d-aff0-944b08f895e6 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:47.985122 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e71563-4730-475d-aff0-944b08f895e6 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:47.985401 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e71563-4730-475d-aff0-944b08f895e6 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:47.990517 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 683/1385] 10.222.0.70 () {54 vars in 912 bytes} [Thu Oct 27 09:36:47 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:36:47.994356 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a5dc6-f715-426e-b52d-a407317da46b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:47.994649 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a5dc6-f715-426e-b52d-a407317da46b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:47.994921 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a5dc6-f715-426e-b52d-a407317da46b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:48.024577 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511935d9-b03c-4996-8795-3208f0370de8 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:48.030014 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb1405-af74-48a0-ad0b-d2bc29a7a979 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:48.030746 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb1405-af74-48a0-ad0b-d2bc29a7a979 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:48.031210 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb1405-af74-48a0-ad0b-d2bc29a7a979 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:48.031592 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb1405-af74-48a0-ad0b-d2bc29a7a979 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:48.031928 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-8b4a5dc6-f715-426e-b52d-a407317da46b None None] MFA Rules not processed for user `45d8920d9a7940abb112dbbb3d7f6e65`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:48.033432 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8b4a5dc6-f715-426e-b52d-a407317da46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:48.049006 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 684/1386] 10.222.0.70 () {58 vars in 1009 bytes} [Thu Oct 27 09:36:47 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:36:48.057280 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcb1405-af74-48a0-ad0b-d2bc29a7a979 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:48.058908 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcb1405-af74-48a0-ad0b-d2bc29a7a979 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:48.071317 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 703/1387] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:36:47 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 09:36:48.137100 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4a5dc6-f715-426e-b52d-a407317da46b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:48.147311 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4a5dc6-f715-426e-b52d-a407317da46b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:48.175364 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb1405-af74-48a0-ad0b-d2bc29a7a979 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:48.177859 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea474105-a2e6-41ba-8665-9f56da7cafda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:48.178609 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea474105-a2e6-41ba-8665-9f56da7cafda service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:48.178889 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea474105-a2e6-41ba-8665-9f56da7cafda service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:48.179164 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea474105-a2e6-41ba-8665-9f56da7cafda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:48.191118 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea474105-a2e6-41ba-8665-9f56da7cafda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:48.193025 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea474105-a2e6-41ba-8665-9f56da7cafda service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:48.210309 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 685/1388] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:36:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:36:48.351048 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb1405-af74-48a0-ad0b-d2bc29a7a979 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:48.368310 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a41cb1-3c0f-4043-8ec6-6af6a93ffcf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:48.369428 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a41cb1-3c0f-4043-8ec6-6af6a93ffcf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:48.369913 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a41cb1-3c0f-4043-8ec6-6af6a93ffcf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:48.370386 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a41cb1-3c0f-4043-8ec6-6af6a93ffcf4 admin admin] PATH_INFO: `/v3/users/bff29d778f5d48548aad0ef6bd47eea3` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:48.371925 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a41cb1-3c0f-4043-8ec6-6af6a93ffcf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bff29d778f5d48548aad0ef6bd47eea3)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:48.377157 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a41cb1-3c0f-4043-8ec6-6af6a93ffcf4 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:48.493763 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a7a41cb1-3c0f-4043-8ec6-6af6a93ffcf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bff29d778f5d48548aad0ef6bd47eea3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:48.528676 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a7a41cb1-3c0f-4043-8ec6-6af6a93ffcf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bff29d778f5d48548aad0ef6bd47eea3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:48.574524 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a7a41cb1-3c0f-4043-8ec6-6af6a93ffcf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bff29d778f5d48548aad0ef6bd47eea3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:48.618030 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a7a41cb1-3c0f-4043-8ec6-6af6a93ffcf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bff29d778f5d48548aad0ef6bd47eea3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:48.619050 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a7a41cb1-3c0f-4043-8ec6-6af6a93ffcf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bff29d778f5d48548aad0ef6bd47eea3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:48.638693 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 704/1389] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:36:48 2022] DELETE /identity/v3/users/bff29d778f5d48548aad0ef6bd47eea3 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:36:48.643277 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9239a8-4a16-40c6-b78a-186a57a8b86d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:48.643797 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9239a8-4a16-40c6-b78a-186a57a8b86d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:48.644315 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9239a8-4a16-40c6-b78a-186a57a8b86d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:48.691953 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-8a9239a8-4a16-40c6-b78a-186a57a8b86d None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:48.698639 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8a9239a8-4a16-40c6-b78a-186a57a8b86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:48.895549 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:48.926959 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 686/1390] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:48.934207 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a41cb1-3c0f-4043-8ec6-6af6a93ffcf4 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:48.935750 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a7a41cb1-3c0f-4043-8ec6-6af6a93ffcf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:48.958116 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:49.143837 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a41cb1-3c0f-4043-8ec6-6af6a93ffcf4 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:49.155658 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb6109-784c-4dc5-95b4-2ea4c4eccbce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:49.156840 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb6109-784c-4dc5-95b4-2ea4c4eccbce service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:49.157286 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb6109-784c-4dc5-95b4-2ea4c4eccbce service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:49.157711 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb6109-784c-4dc5-95b4-2ea4c4eccbce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:49.195064 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fb6109-784c-4dc5-95b4-2ea4c4eccbce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:49.196930 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fb6109-784c-4dc5-95b4-2ea4c4eccbce service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:49.215388 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 705/1391] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:36:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 09:36:49.253298 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a963a5bb-0d2e-4fad-8558-250640ea3bff None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:49.253298 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a963a5bb-0d2e-4fad-8558-250640ea3bff None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:49.253597 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a963a5bb-0d2e-4fad-8558-250640ea3bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:49.306076 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a963a5bb-0d2e-4fad-8558-250640ea3bff None None] MFA Rules not processed for user `2dea51c29cbd46e8bd1454e10eaac012`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:49.311182 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a963a5bb-0d2e-4fad-8558-250640ea3bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:49.485096 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 687/1392] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:49 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:49.534486 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb6109-784c-4dc5-95b4-2ea4c4eccbce service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:49.536828 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c0fb6109-784c-4dc5-95b4-2ea4c4eccbce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:49.667852 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb6109-784c-4dc5-95b4-2ea4c4eccbce service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:49.674032 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269a743a-8fc8-41d5-b8b7-5e8046579afc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2316b30a6e7b4175bd7b90583584eab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:49.674664 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a743a-8fc8-41d5-b8b7-5e8046579afc service placement] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:49.675278 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a743a-8fc8-41d5-b8b7-5e8046579afc service placement] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:49.675560 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a743a-8fc8-41d5-b8b7-5e8046579afc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:49.678155 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-269a743a-8fc8-41d5-b8b7-5e8046579afc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:49.693534 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269a743a-8fc8-41d5-b8b7-5e8046579afc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:49.695175 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269a743a-8fc8-41d5-b8b7-5e8046579afc service placement] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:49.723497 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 706/1393] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:36:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:36:49.877633 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644cc830-a71a-42c2-b48c-cb5f5e44dafb None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:49.878108 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644cc830-a71a-42c2-b48c-cb5f5e44dafb None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:49.878194 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644cc830-a71a-42c2-b48c-cb5f5e44dafb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:49.960823 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-644cc830-a71a-42c2-b48c-cb5f5e44dafb None None] MFA Rules not processed for user `20c65ed67cd8403489949c4dea572d01`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:49.964360 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-644cc830-a71a-42c2-b48c-cb5f5e44dafb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:50.096871 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 688/1394] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:49 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:50.334266 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269a743a-8fc8-41d5-b8b7-5e8046579afc service placement] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:50.335813 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-269a743a-8fc8-41d5-b8b7-5e8046579afc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:50.363081 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269a743a-8fc8-41d5-b8b7-5e8046579afc service placement] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:50.374228 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4362cdc-f8d5-4233-940f-6b77f90c81ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:50.376865 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4362cdc-f8d5-4233-940f-6b77f90c81ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:50.377153 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4362cdc-f8d5-4233-940f-6b77f90c81ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:50.377431 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4362cdc-f8d5-4233-940f-6b77f90c81ba admin admin] PATH_INFO: `/v3/projects/836ce9e6021a4fcd87c892a094d5a1a4` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:50.380511 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4362cdc-f8d5-4233-940f-6b77f90c81ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=836ce9e6021a4fcd87c892a094d5a1a4)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:50.382554 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4362cdc-f8d5-4233-940f-6b77f90c81ba admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:50.572059 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a4362cdc-f8d5-4233-940f-6b77f90c81ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '836ce9e6021a4fcd87c892a094d5a1a4', 'request_id': 'req-a4362cdc-f8d5-4233-940f-6b77f90c81ba'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:50.588718 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a4362cdc-f8d5-4233-940f-6b77f90c81ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '836ce9e6021a4fcd87c892a094d5a1a4', 'request_id': 'req-a4362cdc-f8d5-4233-940f-6b77f90c81ba'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:50.611488 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a4362cdc-f8d5-4233-940f-6b77f90c81ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '836ce9e6021a4fcd87c892a094d5a1a4', 'request_id': 'req-a4362cdc-f8d5-4233-940f-6b77f90c81ba'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:50.630134 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a4362cdc-f8d5-4233-940f-6b77f90c81ba admin admin] The token cache is being invalidate because project 836ce9e6021a4fcd87c892a094d5a1a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:36:50.630715 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a4362cdc-f8d5-4233-940f-6b77f90c81ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:50.632119 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a4362cdc-f8d5-4233-940f-6b77f90c81ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:50.638206 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644cc830-a71a-42c2-b48c-cb5f5e44dafb None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:50.638948 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 707/1395] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:36:50 2022] DELETE /identity/v3/projects/836ce9e6021a4fcd87c892a094d5a1a4 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:36:50.650925 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644cc830-a71a-42c2-b48c-cb5f5e44dafb None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:50.808878 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d075252-986a-4719-bcc8-65558ae6ca83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:50.808878 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d075252-986a-4719-bcc8-65558ae6ca83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:50.808878 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d075252-986a-4719-bcc8-65558ae6ca83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:50.808878 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d075252-986a-4719-bcc8-65558ae6ca83 admin admin] PATH_INFO: `/v3/users/43175f3f693740cb8a0fb8f43d91c8f9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:50.814827 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d075252-986a-4719-bcc8-65558ae6ca83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43175f3f693740cb8a0fb8f43d91c8f9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:50.817263 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d075252-986a-4719-bcc8-65558ae6ca83 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:50.941983 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9d075252-986a-4719-bcc8-65558ae6ca83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43175f3f693740cb8a0fb8f43d91c8f9'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:50.944584 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:50.953507 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9d075252-986a-4719-bcc8-65558ae6ca83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43175f3f693740cb8a0fb8f43d91c8f9'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:50.985013 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9d075252-986a-4719-bcc8-65558ae6ca83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43175f3f693740cb8a0fb8f43d91c8f9'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:51.000633 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9d075252-986a-4719-bcc8-65558ae6ca83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43175f3f693740cb8a0fb8f43d91c8f9'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:51.001703 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9d075252-986a-4719-bcc8-65558ae6ca83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43175f3f693740cb8a0fb8f43d91c8f9'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:51.012005 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 689/1396] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:36:50 2022] DELETE /identity/v3/users/43175f3f693740cb8a0fb8f43d91c8f9 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:36:51.739410 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4362cdc-f8d5-4233-940f-6b77f90c81ba admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:51.742472 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a4362cdc-f8d5-4233-940f-6b77f90c81ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:51.769143 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4362cdc-f8d5-4233-940f-6b77f90c81ba admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:51.854805 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:51.983053 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ef7fa3-08d1-4ce5-9e01-94f0a0e95d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:51.988833 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ef7fa3-08d1-4ce5-9e01-94f0a0e95d5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:51.989135 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ef7fa3-08d1-4ce5-9e01-94f0a0e95d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:51.989413 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ef7fa3-08d1-4ce5-9e01-94f0a0e95d5a admin admin] PATH_INFO: `/v3/projects/f2946c1d27dc41378f2453e89b11fbd5` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:51.993460 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ef7fa3-08d1-4ce5-9e01-94f0a0e95d5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2946c1d27dc41378f2453e89b11fbd5)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:52.005681 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ef7fa3-08d1-4ce5-9e01-94f0a0e95d5a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:52.090376 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e4ef7fa3-08d1-4ce5-9e01-94f0a0e95d5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2946c1d27dc41378f2453e89b11fbd5', 'request_id': 'req-e4ef7fa3-08d1-4ce5-9e01-94f0a0e95d5a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:52.102374 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e4ef7fa3-08d1-4ce5-9e01-94f0a0e95d5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2946c1d27dc41378f2453e89b11fbd5', 'request_id': 'req-e4ef7fa3-08d1-4ce5-9e01-94f0a0e95d5a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:52.124227 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e4ef7fa3-08d1-4ce5-9e01-94f0a0e95d5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2946c1d27dc41378f2453e89b11fbd5', 'request_id': 'req-e4ef7fa3-08d1-4ce5-9e01-94f0a0e95d5a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:52.143562 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e4ef7fa3-08d1-4ce5-9e01-94f0a0e95d5a admin admin] The token cache is being invalidate because project f2946c1d27dc41378f2453e89b11fbd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:36:52.145446 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e4ef7fa3-08d1-4ce5-9e01-94f0a0e95d5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:52.146833 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e4ef7fa3-08d1-4ce5-9e01-94f0a0e95d5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:52.155082 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 708/1397] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:36:51 2022] DELETE /identity/v3/projects/f2946c1d27dc41378f2453e89b11fbd5 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:36:52.160723 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61b0462-82ad-4472-9b11-05a04fa14138 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:52.161602 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61b0462-82ad-4472-9b11-05a04fa14138 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:52.162044 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61b0462-82ad-4472-9b11-05a04fa14138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:52.209876 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a61b0462-82ad-4472-9b11-05a04fa14138 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:52.211446 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a61b0462-82ad-4472-9b11-05a04fa14138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:52.492848 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 690/1398] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:52.501313 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202bbab-6f86-4b0e-a37e-afc12b15e862 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:52.501600 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202bbab-6f86-4b0e-a37e-afc12b15e862 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:52.501869 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202bbab-6f86-4b0e-a37e-afc12b15e862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:52.553230 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61b0462-82ad-4472-9b11-05a04fa14138 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:52.555678 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a61b0462-82ad-4472-9b11-05a04fa14138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:52.557653 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-9202bbab-6f86-4b0e-a37e-afc12b15e862 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:52.559293 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9202bbab-6f86-4b0e-a37e-afc12b15e862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:52.579762 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61b0462-82ad-4472-9b11-05a04fa14138 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:52.586113 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 709/1399] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:52.591084 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9202bbab-6f86-4b0e-a37e-afc12b15e862 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:52.591445 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586c3959-08d8-4d17-8881-a1484ce6b3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:52.592475 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c3959-08d8-4d17-8881-a1484ce6b3aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:52.592922 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c3959-08d8-4d17-8881-a1484ce6b3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:52.594596 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c3959-08d8-4d17-8881-a1484ce6b3aa admin admin] PATH_INFO: `/v3/users/7056c7a20375483f89d5a535188a4a7b` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:52.595993 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586c3959-08d8-4d17-8881-a1484ce6b3aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=7056c7a20375483f89d5a535188a4a7b)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:52.604938 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9202bbab-6f86-4b0e-a37e-afc12b15e862 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:52.618035 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586c3959-08d8-4d17-8881-a1484ce6b3aa admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:52.630886 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e980c6-f70e-4af1-866f-2e42eff13004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:52.631625 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e980c6-f70e-4af1-866f-2e42eff13004 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:52.631795 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e980c6-f70e-4af1-866f-2e42eff13004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:52.632017 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e980c6-f70e-4af1-866f-2e42eff13004 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:52.632728 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e980c6-f70e-4af1-866f-2e42eff13004 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:36:52.632993 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e980c6-f70e-4af1-866f-2e42eff13004 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:52.634632 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e980c6-f70e-4af1-866f-2e42eff13004 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:52.670694 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 710/1400] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:36:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:36:52.677591 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e980c6-f70e-4af1-866f-2e42eff13004 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:52.693728 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e980c6-f70e-4af1-866f-2e42eff13004 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:52.706725 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a589fb-bfb6-43bc-9335-a7322baab8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:52.708137 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a589fb-bfb6-43bc-9335-a7322baab8bc admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:52.709692 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a589fb-bfb6-43bc-9335-a7322baab8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:52.709979 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a589fb-bfb6-43bc-9335-a7322baab8bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:52.710676 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a589fb-bfb6-43bc-9335-a7322baab8bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:52.716242 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a589fb-bfb6-43bc-9335-a7322baab8bc admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:52.741818 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-586c3959-08d8-4d17-8881-a1484ce6b3aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7056c7a20375483f89d5a535188a4a7b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:52.772086 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-586c3959-08d8-4d17-8881-a1484ce6b3aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7056c7a20375483f89d5a535188a4a7b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:52.819085 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-586c3959-08d8-4d17-8881-a1484ce6b3aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7056c7a20375483f89d5a535188a4a7b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:52.839218 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-586c3959-08d8-4d17-8881-a1484ce6b3aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7056c7a20375483f89d5a535188a4a7b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:52.840310 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-586c3959-08d8-4d17-8881-a1484ce6b3aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7056c7a20375483f89d5a535188a4a7b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:52.857230 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 711/1401] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:36:52 2022] POST /identity/v3/projects => generated 393 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:36:52.863788 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a589fb-bfb6-43bc-9335-a7322baab8bc admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:52.869049 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b7a589fb-bfb6-43bc-9335-a7322baab8bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:52.872234 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 691/1402] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:36:52 2022] DELETE /identity/v3/users/7056c7a20375483f89d5a535188a4a7b => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:36:52.878310 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c799b-9d3d-4655-9ca6-ab70f591a8a1 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:52.878597 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c799b-9d3d-4655-9ca6-ab70f591a8a1 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:52.878872 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c799b-9d3d-4655-9ca6-ab70f591a8a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:52.949916 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-217c799b-9d3d-4655-9ca6-ab70f591a8a1 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:52.961622 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-217c799b-9d3d-4655-9ca6-ab70f591a8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:53.100460 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a589fb-bfb6-43bc-9335-a7322baab8bc admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:53.164815 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:53.187927 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:53.299157 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 692/1403] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:53.300747 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98d64e-cc54-4e37-830b-abd61c40f0c8 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:53.300747 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98d64e-cc54-4e37-830b-abd61c40f0c8 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:53.301199 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98d64e-cc54-4e37-830b-abd61c40f0c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:53.315467 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db971083-a9b6-44b7-b76a-f6077d489b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:53.316291 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db971083-a9b6-44b7-b76a-f6077d489b68 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:53.316617 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db971083-a9b6-44b7-b76a-f6077d489b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:53.317027 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db971083-a9b6-44b7-b76a-f6077d489b68 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:53.317667 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db971083-a9b6-44b7-b76a-f6077d489b68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:53.319885 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db971083-a9b6-44b7-b76a-f6077d489b68 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:53.376262 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 712/1404] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:36:52 2022] POST /identity/v3/users => generated 297 bytes in 509 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:36:53.378168 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db971083-a9b6-44b7-b76a-f6077d489b68 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:53.401640 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db971083-a9b6-44b7-b76a-f6077d489b68 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:53.408675 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ccd39-19af-49e0-a05c-028a0d767727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:53.409301 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ccd39-19af-49e0-a05c-028a0d767727 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:53.409579 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ccd39-19af-49e0-a05c-028a0d767727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:53.409850 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ccd39-19af-49e0-a05c-028a0d767727 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:53.411452 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7ccd39-19af-49e0-a05c-028a0d767727 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:53.413544 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7ccd39-19af-49e0-a05c-028a0d767727 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:53.454763 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 713/1405] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 09:36:53.454763 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ccd39-19af-49e0-a05c-028a0d767727 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:53.464935 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-3b98d64e-cc54-4e37-830b-abd61c40f0c8 None None] MFA Rules not processed for user `7f389510a6a044d8a7487561b169544b`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:53.465133 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4e7ccd39-19af-49e0-a05c-028a0d767727 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:53.468589 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3b98d64e-cc54-4e37-830b-abd61c40f0c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:53.666725 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ccd39-19af-49e0-a05c-028a0d767727 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:53.673538 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 693/1406] 10.222.0.70 () {58 vars in 1009 bytes} [Thu Oct 27 09:36:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:53.674312 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b98d64e-cc54-4e37-830b-abd61c40f0c8 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:53.689734 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ff07f6-f4b4-4425-89ea-b4acd783c66f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:53.690617 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff07f6-f4b4-4425-89ea-b4acd783c66f service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:53.691109 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff07f6-f4b4-4425-89ea-b4acd783c66f service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:53.691593 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff07f6-f4b4-4425-89ea-b4acd783c66f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:53.701206 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b98d64e-cc54-4e37-830b-abd61c40f0c8 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:53.709840 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea4e469-7253-4ce5-94c5-57bf1064e4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:53.710797 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea4e469-7253-4ce5-94c5-57bf1064e4e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:53.711550 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea4e469-7253-4ce5-94c5-57bf1064e4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:53.714968 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ff07f6-f4b4-4425-89ea-b4acd783c66f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:53.718498 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ff07f6-f4b4-4425-89ea-b4acd783c66f service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:53.720834 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea4e469-7253-4ce5-94c5-57bf1064e4e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:53.722602 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea4e469-7253-4ce5-94c5-57bf1064e4e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:53.725292 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea4e469-7253-4ce5-94c5-57bf1064e4e4 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:53.752221 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 714/1407] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:36:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:36:53.762440 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 694/1408] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:36:53.767301 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ff07f6-f4b4-4425-89ea-b4acd783c66f service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:53.779385 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ff07f6-f4b4-4425-89ea-b4acd783c66f service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:53.804785 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607a425a-a642-445b-a57c-75e7a79bc9af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:53.805535 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a425a-a642-445b-a57c-75e7a79bc9af admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:53.805708 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a425a-a642-445b-a57c-75e7a79bc9af admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:53.805980 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a425a-a642-445b-a57c-75e7a79bc9af admin admin] PATH_INFO: `/v3/projects/0c43be6e1ab24eea884e9f3514253828/users/6c295d2a0cf34c268a67e891bb0b709c/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:53.823708 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607a425a-a642-445b-a57c-75e7a79bc9af admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c43be6e1ab24eea884e9f3514253828, user_id=6c295d2a0cf34c268a67e891bb0b709c, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:53.827037 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607a425a-a642-445b-a57c-75e7a79bc9af admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:53.864588 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 715/1409] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:36:53 2022] PUT /identity/v3/projects/0c43be6e1ab24eea884e9f3514253828/users/6c295d2a0cf34c268a67e891bb0b709c/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:36:53.866198 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d1dfc-8aa9-451d-b075-1300a425aedc None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:53.866491 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d1dfc-8aa9-451d-b075-1300a425aedc None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:53.866912 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d1dfc-8aa9-451d-b075-1300a425aedc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:53.943527 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607a425a-a642-445b-a57c-75e7a79bc9af admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:53.945314 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-607a425a-a642-445b-a57c-75e7a79bc9af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:54.028582 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-322d1dfc-8aa9-451d-b075-1300a425aedc None None] MFA Rules not processed for user `6c295d2a0cf34c268a67e891bb0b709c`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:54.036216 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-322d1dfc-8aa9-451d-b075-1300a425aedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:54.258285 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 695/1410] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:53 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 393 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:36:54.260413 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d1dfc-8aa9-451d-b075-1300a425aedc None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:54.290946 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:54.317786 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607a425a-a642-445b-a57c-75e7a79bc9af admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:54.328486 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:54.332882 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803000b7-9944-4f32-93d1-45fbc3b1af35 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eecf79ea4700496c89b1a67c614b34d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:54.336206 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803000b7-9944-4f32-93d1-45fbc3b1af35 service swift] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:54.336206 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803000b7-9944-4f32-93d1-45fbc3b1af35 service swift] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:54.336206 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803000b7-9944-4f32-93d1-45fbc3b1af35 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:54.412608 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d1dfc-8aa9-451d-b075-1300a425aedc None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:54.425724 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43730f-27c2-4ab1-9a20-fc887fb3682e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:54.426614 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43730f-27c2-4ab1-9a20-fc887fb3682e admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:54.427925 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43730f-27c2-4ab1-9a20-fc887fb3682e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:54.428366 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43730f-27c2-4ab1-9a20-fc887fb3682e admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:54.437450 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b43730f-27c2-4ab1-9a20-fc887fb3682e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:54.437450 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b43730f-27c2-4ab1-9a20-fc887fb3682e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:54.525013 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 696/1411] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:36:54 2022] POST /identity/v3/projects => generated 393 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:36:54.526795 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43730f-27c2-4ab1-9a20-fc887fb3682e admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:54.538590 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803000b7-9944-4f32-93d1-45fbc3b1af35 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:54.544383 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803000b7-9944-4f32-93d1-45fbc3b1af35 service swift] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:54.584837 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 716/1412] 10.222.0.70 () {60 vars in 1384 bytes} [Thu Oct 27 09:36:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 641 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 27 09:36:54.709794 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43730f-27c2-4ab1-9a20-fc887fb3682e admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:54.715237 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1082174-7e12-4fa8-aefd-e0d49c88dbab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:54.715986 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1082174-7e12-4fa8-aefd-e0d49c88dbab admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:54.716369 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1082174-7e12-4fa8-aefd-e0d49c88dbab admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:54.716773 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1082174-7e12-4fa8-aefd-e0d49c88dbab admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:54.717512 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1082174-7e12-4fa8-aefd-e0d49c88dbab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:54.720288 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1082174-7e12-4fa8-aefd-e0d49c88dbab admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:54.781815 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803000b7-9944-4f32-93d1-45fbc3b1af35 service swift] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:54.789263 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 697/1413] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:36:54 2022] POST /identity/v3/users => generated 303 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:36:54.797961 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803000b7-9944-4f32-93d1-45fbc3b1af35 service swift] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:54.849077 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e4defa-7cbf-4804-a6eb-5736453a2bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:54.849935 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e4defa-7cbf-4804-a6eb-5736453a2bb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:54.850428 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e4defa-7cbf-4804-a6eb-5736453a2bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:54.851113 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e4defa-7cbf-4804-a6eb-5736453a2bb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:54.852111 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e4defa-7cbf-4804-a6eb-5736453a2bb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:54.857317 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e4defa-7cbf-4804-a6eb-5736453a2bb2 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:54.882871 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1082174-7e12-4fa8-aefd-e0d49c88dbab admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:54.888605 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 717/1414] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:36:54.911596 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1082174-7e12-4fa8-aefd-e0d49c88dbab admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:54.924187 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa115020-eb0d-49cc-a28c-3d88417631e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:54.926298 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa115020-eb0d-49cc-a28c-3d88417631e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:54.926753 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa115020-eb0d-49cc-a28c-3d88417631e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:54.927333 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa115020-eb0d-49cc-a28c-3d88417631e6 admin admin] PATH_INFO: `/v3/projects/8a23c871c2204fdbaeef2619371276f0/users/6f68bbb911c4470dbd085771c806634f/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:54.953898 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa115020-eb0d-49cc-a28c-3d88417631e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a23c871c2204fdbaeef2619371276f0, user_id=6f68bbb911c4470dbd085771c806634f, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:54.957687 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa115020-eb0d-49cc-a28c-3d88417631e6 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:54.994628 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 698/1415] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:36:54 2022] PUT /identity/v3/projects/8a23c871c2204fdbaeef2619371276f0/users/6f68bbb911c4470dbd085771c806634f/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:36:55.000944 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e4defa-7cbf-4804-a6eb-5736453a2bb2 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:55.024183 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e4defa-7cbf-4804-a6eb-5736453a2bb2 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:55.162454 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa115020-eb0d-49cc-a28c-3d88417631e6 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:55.165154 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-aa115020-eb0d-49cc-a28c-3d88417631e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:55.427095 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f99c3d2-c054-4001-b286-35f35002b632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:55.427727 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99c3d2-c054-4001-b286-35f35002b632 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:55.428205 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99c3d2-c054-4001-b286-35f35002b632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:55.428444 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99c3d2-c054-4001-b286-35f35002b632 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:55.429116 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f99c3d2-c054-4001-b286-35f35002b632 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:55.430732 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f99c3d2-c054-4001-b286-35f35002b632 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:55.469565 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 718/1416] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 470 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:36:55.479083 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f99c3d2-c054-4001-b286-35f35002b632 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:55.498417 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa115020-eb0d-49cc-a28c-3d88417631e6 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:55.503627 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f99c3d2-c054-4001-b286-35f35002b632 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:55.520345 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da840a29-bf52-4ed0-9918-b9db73f16396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:55.522461 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da840a29-bf52-4ed0-9918-b9db73f16396 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:55.523613 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da840a29-bf52-4ed0-9918-b9db73f16396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:55.524414 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6632c6-c518-427d-9c8b-5e3683787da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:55.524928 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da840a29-bf52-4ed0-9918-b9db73f16396 admin admin] PATH_INFO: `/v3/users/4cc4ce3e1fa748acaaa5ec8864c6e73f` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:55.526789 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6632c6-c518-427d-9c8b-5e3683787da2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:55.527685 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da840a29-bf52-4ed0-9918-b9db73f16396 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cc4ce3e1fa748acaaa5ec8864c6e73f)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:55.528162 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6632c6-c518-427d-9c8b-5e3683787da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:55.529423 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6632c6-c518-427d-9c8b-5e3683787da2 admin admin] PATH_INFO: `/v3/projects/8a23c871c2204fdbaeef2619371276f0/users/6f68bbb911c4470dbd085771c806634f/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:55.539058 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da840a29-bf52-4ed0-9918-b9db73f16396 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:55.549793 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6632c6-c518-427d-9c8b-5e3683787da2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a23c871c2204fdbaeef2619371276f0, user_id=6f68bbb911c4470dbd085771c806634f, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:55.562518 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6632c6-c518-427d-9c8b-5e3683787da2 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:55.596438 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 719/1417] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:36:55 2022] PUT /identity/v3/projects/8a23c871c2204fdbaeef2619371276f0/users/6f68bbb911c4470dbd085771c806634f/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:36:55.608165 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1e362-7580-4825-be22-ba1950d121c0 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:55.609001 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:55.610400 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1e362-7580-4825-be22-ba1950d121c0 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:55.610958 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1e362-7580-4825-be22-ba1950d121c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:55.659486 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-da840a29-bf52-4ed0-9918-b9db73f16396 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cc4ce3e1fa748acaaa5ec8864c6e73f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:55.680184 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-da840a29-bf52-4ed0-9918-b9db73f16396 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cc4ce3e1fa748acaaa5ec8864c6e73f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:55.683366 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f8e1e362-7580-4825-be22-ba1950d121c0 None None] MFA Rules not processed for user `6f68bbb911c4470dbd085771c806634f`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:55.685907 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f8e1e362-7580-4825-be22-ba1950d121c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:55.717660 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-da840a29-bf52-4ed0-9918-b9db73f16396 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cc4ce3e1fa748acaaa5ec8864c6e73f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:55.739127 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-da840a29-bf52-4ed0-9918-b9db73f16396 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cc4ce3e1fa748acaaa5ec8864c6e73f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:55.740314 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-da840a29-bf52-4ed0-9918-b9db73f16396 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cc4ce3e1fa748acaaa5ec8864c6e73f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:55.751950 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 699/1418] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:36:55 2022] DELETE /identity/v3/users/4cc4ce3e1fa748acaaa5ec8864c6e73f => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:36:55.758938 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b814313f-ef46-45ec-95a9-9c4ca95a96e0 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:55.759217 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b814313f-ef46-45ec-95a9-9c4ca95a96e0 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:55.759499 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b814313f-ef46-45ec-95a9-9c4ca95a96e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:55.835920 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b814313f-ef46-45ec-95a9-9c4ca95a96e0 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:55.837866 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:55.854003 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b814313f-ef46-45ec-95a9-9c4ca95a96e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:55.887548 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 720/1419] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:55 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:36:55.889030 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e1e362-7580-4825-be22-ba1950d121c0 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:55.895507 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f8e1e362-7580-4825-be22-ba1950d121c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:56.071645 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 700/1420] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:56.073014 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b814313f-ef46-45ec-95a9-9c4ca95a96e0 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:56.076179 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b814313f-ef46-45ec-95a9-9c4ca95a96e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:56.076608 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e1e362-7580-4825-be22-ba1950d121c0 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:56.082293 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbaabeb-6c93-4346-8825-cac607d9382a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:56.082954 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbaabeb-6c93-4346-8825-cac607d9382a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:56.083258 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbaabeb-6c93-4346-8825-cac607d9382a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:56.083532 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbaabeb-6c93-4346-8825-cac607d9382a admin admin] PATH_INFO: `/v3/projects/c393eb39d1ca42aa879756bb8d7ec4e3` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:56.085083 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbaabeb-6c93-4346-8825-cac607d9382a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c393eb39d1ca42aa879756bb8d7ec4e3)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:56.087947 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbaabeb-6c93-4346-8825-cac607d9382a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:56.228295 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9dbaabeb-6c93-4346-8825-cac607d9382a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c393eb39d1ca42aa879756bb8d7ec4e3', 'request_id': 'req-9dbaabeb-6c93-4346-8825-cac607d9382a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:56.240265 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b814313f-ef46-45ec-95a9-9c4ca95a96e0 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:56.261149 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9dbaabeb-6c93-4346-8825-cac607d9382a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c393eb39d1ca42aa879756bb8d7ec4e3', 'request_id': 'req-9dbaabeb-6c93-4346-8825-cac607d9382a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:56.312025 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9dbaabeb-6c93-4346-8825-cac607d9382a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c393eb39d1ca42aa879756bb8d7ec4e3', 'request_id': 'req-9dbaabeb-6c93-4346-8825-cac607d9382a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:56.352947 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9dbaabeb-6c93-4346-8825-cac607d9382a admin admin] The token cache is being invalidate because project c393eb39d1ca42aa879756bb8d7ec4e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:36:56.355694 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9dbaabeb-6c93-4346-8825-cac607d9382a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:56.358412 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9dbaabeb-6c93-4346-8825-cac607d9382a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:56.360480 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 721/1421] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:36:55 2022] DELETE /identity/v3/projects/c393eb39d1ca42aa879756bb8d7ec4e3 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:36:56.361237 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbaabeb-6c93-4346-8825-cac607d9382a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:56.363933 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9dbaabeb-6c93-4346-8825-cac607d9382a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:56.434715 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e7926f-ffec-45f7-adfe-5c4cddcb6a97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:56.435642 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7926f-ffec-45f7-adfe-5c4cddcb6a97 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:56.436006 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7926f-ffec-45f7-adfe-5c4cddcb6a97 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:56.436418 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7926f-ffec-45f7-adfe-5c4cddcb6a97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:56.459938 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e7e7926f-ffec-45f7-adfe-5c4cddcb6a97 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:56.486590 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:56.564364 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbaabeb-6c93-4346-8825-cac607d9382a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:56.567231 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa7729-32f3-4bd0-a4aa-c60d19eb5a1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:56.568474 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa7729-32f3-4bd0-a4aa-c60d19eb5a1c service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:56.569087 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa7729-32f3-4bd0-a4aa-c60d19eb5a1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:56.569333 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa7729-32f3-4bd0-a4aa-c60d19eb5a1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:56.571434 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b5aa7729-32f3-4bd0-a4aa-c60d19eb5a1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:56.615486 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e7926f-ffec-45f7-adfe-5c4cddcb6a97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:56.617791 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e7926f-ffec-45f7-adfe-5c4cddcb6a97 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:56.647561 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 701/1422] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:36:56 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 574 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:36:56.652254 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30bd71e-b106-4c9a-906c-41640651676c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:56.652740 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30bd71e-b106-4c9a-906c-41640651676c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:56.653128 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30bd71e-b106-4c9a-906c-41640651676c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:56.690770 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:56.707765 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-c30bd71e-b106-4c9a-906c-41640651676c None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:56.709272 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c30bd71e-b106-4c9a-906c-41640651676c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:56.778879 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aa7729-32f3-4bd0-a4aa-c60d19eb5a1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:56.780376 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aa7729-32f3-4bd0-a4aa-c60d19eb5a1c service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:56.807444 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 722/1423] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:36:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:36:56.808314 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa7729-32f3-4bd0-a4aa-c60d19eb5a1c service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:56.818690 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b5aa7729-32f3-4bd0-a4aa-c60d19eb5a1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:56.852816 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa7729-32f3-4bd0-a4aa-c60d19eb5a1c service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:56.868638 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bbdfa5-3169-4372-924f-564575e8d10d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:56.869536 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bbdfa5-3169-4372-924f-564575e8d10d service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:56.870031 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bbdfa5-3169-4372-924f-564575e8d10d service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:56.870505 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bbdfa5-3169-4372-924f-564575e8d10d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:56.875477 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 702/1424] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:56.881530 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-35bbdfa5-3169-4372-924f-564575e8d10d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:56.893977 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa9d637-5af5-4e27-81f4-6bb5afa72809 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:56.895153 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa9d637-5af5-4e27-81f4-6bb5afa72809 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:56.896890 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa9d637-5af5-4e27-81f4-6bb5afa72809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:56.957773 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-9fa9d637-5af5-4e27-81f4-6bb5afa72809 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:56.962084 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9fa9d637-5af5-4e27-81f4-6bb5afa72809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:56.987018 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 703/1425] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:56.991685 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa9d637-5af5-4e27-81f4-6bb5afa72809 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:57.011858 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa9d637-5af5-4e27-81f4-6bb5afa72809 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:57.016429 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bbdfa5-3169-4372-924f-564575e8d10d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:57.018040 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bbdfa5-3169-4372-924f-564575e8d10d service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:57.023199 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0339b7-3674-442a-8fbf-10343d79a298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:57.023861 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0339b7-3674-442a-8fbf-10343d79a298 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:57.024133 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0339b7-3674-442a-8fbf-10343d79a298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:57.024402 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0339b7-3674-442a-8fbf-10343d79a298 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:57.025255 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0339b7-3674-442a-8fbf-10343d79a298 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:36:57.025462 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0339b7-3674-442a-8fbf-10343d79a298 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:57.027081 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0339b7-3674-442a-8fbf-10343d79a298 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:57.042220 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 704/1426] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:36:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:36:57.042923 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 723/1427] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:36:56 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:36:57.057890 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0339b7-3674-442a-8fbf-10343d79a298 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:57.080364 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0339b7-3674-442a-8fbf-10343d79a298 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:57.095615 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0da1b-2cbe-40fc-9c66-9a2b6f071e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:57.096439 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0da1b-2cbe-40fc-9c66-9a2b6f071e03 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:57.101164 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0da1b-2cbe-40fc-9c66-9a2b6f071e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:57.101474 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0da1b-2cbe-40fc-9c66-9a2b6f071e03 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:57.103290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb0da1b-2cbe-40fc-9c66-9a2b6f071e03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:57.107428 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb0da1b-2cbe-40fc-9c66-9a2b6f071e03 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:57.173054 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 705/1428] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:36:57 2022] POST /identity/v3/projects => generated 401 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:36:57.174634 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bbdfa5-3169-4372-924f-564575e8d10d service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:57.201744 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bbdfa5-3169-4372-924f-564575e8d10d service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:57.377242 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87a0bb-bd5c-40e2-9a87-b68a0a2d59c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:57.378228 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87a0bb-bd5c-40e2-9a87-b68a0a2d59c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:57.378771 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87a0bb-bd5c-40e2-9a87-b68a0a2d59c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:57.379243 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87a0bb-bd5c-40e2-9a87-b68a0a2d59c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:57.380158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c87a0bb-bd5c-40e2-9a87-b68a0a2d59c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:57.382507 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c87a0bb-bd5c-40e2-9a87-b68a0a2d59c4 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:57.489332 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 724/1429] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:36:57 2022] POST /identity/v3/users => generated 301 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:36:57.493129 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0da1b-2cbe-40fc-9c66-9a2b6f071e03 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:57.515411 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0da1b-2cbe-40fc-9c66-9a2b6f071e03 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:57.533513 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62f8764-b902-4582-b9b2-6ec3278ee4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:57.534232 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62f8764-b902-4582-b9b2-6ec3278ee4e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:57.534926 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62f8764-b902-4582-b9b2-6ec3278ee4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:57.546138 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62f8764-b902-4582-b9b2-6ec3278ee4e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:57.546855 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62f8764-b902-4582-b9b2-6ec3278ee4e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:57.548508 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62f8764-b902-4582-b9b2-6ec3278ee4e3 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:57.590639 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 706/1430] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:36:57.592355 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87a0bb-bd5c-40e2-9a87-b68a0a2d59c4 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:57.622670 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87a0bb-bd5c-40e2-9a87-b68a0a2d59c4 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:57.627510 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda6a5e7-fd6f-4946-acec-95387eb10d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:57.628435 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6a5e7-fd6f-4946-acec-95387eb10d1a admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:57.628435 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6a5e7-fd6f-4946-acec-95387eb10d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:57.629823 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6a5e7-fd6f-4946-acec-95387eb10d1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:57.638744 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda6a5e7-fd6f-4946-acec-95387eb10d1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:57.652224 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda6a5e7-fd6f-4946-acec-95387eb10d1a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:57.701353 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 725/1431] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:36:57.706198 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62f8764-b902-4582-b9b2-6ec3278ee4e3 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:57.723538 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62f8764-b902-4582-b9b2-6ec3278ee4e3 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:57.739407 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c200907-a506-476d-9f0b-98ea7276814f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:57.740434 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c200907-a506-476d-9f0b-98ea7276814f admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:57.740434 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c200907-a506-476d-9f0b-98ea7276814f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:57.741234 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c200907-a506-476d-9f0b-98ea7276814f admin admin] PATH_INFO: `/v3/projects/0ce93fdbf7944edc8f3dd37e95f89022/users/1004f2a0315c46638e027c79a071f9cf/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:57.808410 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c200907-a506-476d-9f0b-98ea7276814f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ce93fdbf7944edc8f3dd37e95f89022, user_id=1004f2a0315c46638e027c79a071f9cf, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:57.811337 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c200907-a506-476d-9f0b-98ea7276814f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:57.842159 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 707/1432] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:36:57 2022] PUT /identity/v3/projects/0ce93fdbf7944edc8f3dd37e95f89022/users/1004f2a0315c46638e027c79a071f9cf/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:36:57.853857 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c0312-1338-41fb-b148-0f8e317b6a63 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:57.854013 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c0312-1338-41fb-b148-0f8e317b6a63 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:57.854303 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c0312-1338-41fb-b148-0f8e317b6a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:57.937410 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-581c0312-1338-41fb-b148-0f8e317b6a63 None None] MFA Rules not processed for user `1004f2a0315c46638e027c79a071f9cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:57.938661 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-581c0312-1338-41fb-b148-0f8e317b6a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:58.041580 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:58.079604 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 726/1433] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:57 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:36:58.084046 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c200907-a506-476d-9f0b-98ea7276814f admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:58.136756 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:58.188923 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c200907-a506-476d-9f0b-98ea7276814f admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:58.194783 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581c0312-1338-41fb-b148-0f8e317b6a63 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:58.196197 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-581c0312-1338-41fb-b148-0f8e317b6a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:58.213849 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ef99e-92b9-482d-8384-52448a4dfbec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:58.214491 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ef99e-92b9-482d-8384-52448a4dfbec admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:58.214764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ef99e-92b9-482d-8384-52448a4dfbec admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:58.215032 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ef99e-92b9-482d-8384-52448a4dfbec admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:58.215686 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202ef99e-92b9-482d-8384-52448a4dfbec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:58.218364 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202ef99e-92b9-482d-8384-52448a4dfbec admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:58.231096 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581c0312-1338-41fb-b148-0f8e317b6a63 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:58.241116 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3174356a-aac9-493a-80a2-66fdec23dd0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:58.242412 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174356a-aac9-493a-80a2-66fdec23dd0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:58.243012 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174356a-aac9-493a-80a2-66fdec23dd0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:58.243581 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174356a-aac9-493a-80a2-66fdec23dd0f admin admin] PATH_INFO: `/v3/projects/ad4158b35c7d4afe948732f0ea817257` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:58.246662 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3174356a-aac9-493a-80a2-66fdec23dd0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad4158b35c7d4afe948732f0ea817257)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:58.257176 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3174356a-aac9-493a-80a2-66fdec23dd0f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:58.268254 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 708/1434] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:36:58 2022] POST /identity/v3/projects => generated 405 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:36:58.272268 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ef99e-92b9-482d-8384-52448a4dfbec admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:58.337016 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3174356a-aac9-493a-80a2-66fdec23dd0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad4158b35c7d4afe948732f0ea817257', 'request_id': 'req-3174356a-aac9-493a-80a2-66fdec23dd0f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:58.344721 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3174356a-aac9-493a-80a2-66fdec23dd0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad4158b35c7d4afe948732f0ea817257', 'request_id': 'req-3174356a-aac9-493a-80a2-66fdec23dd0f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:58.357014 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3174356a-aac9-493a-80a2-66fdec23dd0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad4158b35c7d4afe948732f0ea817257', 'request_id': 'req-3174356a-aac9-493a-80a2-66fdec23dd0f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:58.371561 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3174356a-aac9-493a-80a2-66fdec23dd0f admin admin] The token cache is being invalidate because project ad4158b35c7d4afe948732f0ea817257 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:36:58.371961 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3174356a-aac9-493a-80a2-66fdec23dd0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:58.372754 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3174356a-aac9-493a-80a2-66fdec23dd0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:58.378669 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 727/1435] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:36:58 2022] DELETE /identity/v3/projects/ad4158b35c7d4afe948732f0ea817257 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:36:58.381857 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ef99e-92b9-482d-8384-52448a4dfbec admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:58.382235 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3174356a-aac9-493a-80a2-66fdec23dd0f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:58.383732 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3174356a-aac9-493a-80a2-66fdec23dd0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:58.450818 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e96f9b0-bc89-4b69-8406-b048d26a3ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:58.451538 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96f9b0-bc89-4b69-8406-b048d26a3ab2 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:58.451827 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96f9b0-bc89-4b69-8406-b048d26a3ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:58.452102 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96f9b0-bc89-4b69-8406-b048d26a3ab2 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:58.453669 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e96f9b0-bc89-4b69-8406-b048d26a3ab2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:58.456406 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e96f9b0-bc89-4b69-8406-b048d26a3ab2 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:58.462077 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3174356a-aac9-493a-80a2-66fdec23dd0f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:58.467140 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c52930-465c-4170-acaf-b947260c6ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:58.467905 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c52930-465c-4170-acaf-b947260c6ce1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:58.468172 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c52930-465c-4170-acaf-b947260c6ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:58.468467 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c52930-465c-4170-acaf-b947260c6ce1 admin admin] PATH_INFO: `/v3/users/54970c6f86c14078af579866d8e046e0` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:58.469678 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c52930-465c-4170-acaf-b947260c6ce1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54970c6f86c14078af579866d8e046e0)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:58.472200 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c52930-465c-4170-acaf-b947260c6ce1 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:58.492170 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 709/1436] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:36:58 2022] POST /identity/v3/users => generated 309 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:36:58.494220 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e96f9b0-bc89-4b69-8406-b048d26a3ab2 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:58.495773 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9e96f9b0-bc89-4b69-8406-b048d26a3ab2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:58.513038 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e96f9b0-bc89-4b69-8406-b048d26a3ab2 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:58.520053 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b5ac94-8267-4eb6-80d8-03146333ec82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:58.520705 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b5ac94-8267-4eb6-80d8-03146333ec82 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:58.520987 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b5ac94-8267-4eb6-80d8-03146333ec82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:58.521258 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b5ac94-8267-4eb6-80d8-03146333ec82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:58.521924 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b5ac94-8267-4eb6-80d8-03146333ec82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:58.524744 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b5ac94-8267-4eb6-80d8-03146333ec82 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:58.532352 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-04c52930-465c-4170-acaf-b947260c6ce1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54970c6f86c14078af579866d8e046e0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:58.540694 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 710/1437] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:36:58 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 09:36:58.543151 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-04c52930-465c-4170-acaf-b947260c6ce1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54970c6f86c14078af579866d8e046e0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:58.543327 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b5ac94-8267-4eb6-80d8-03146333ec82 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:58.554445 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:58.556754 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b5ac94-8267-4eb6-80d8-03146333ec82 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:58.560049 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-04c52930-465c-4170-acaf-b947260c6ce1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54970c6f86c14078af579866d8e046e0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:58.562933 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e71b31-3b40-4810-86d1-77151c80e523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:58.563839 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e71b31-3b40-4810-86d1-77151c80e523 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:58.564215 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e71b31-3b40-4810-86d1-77151c80e523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:58.564593 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e71b31-3b40-4810-86d1-77151c80e523 admin admin] PATH_INFO: `/v3/projects/a1b5e9e9d18d4235973fdc40e44020a7/users/0766dfad668d41f189c00cfb38389930/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:58.570776 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-04c52930-465c-4170-acaf-b947260c6ce1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54970c6f86c14078af579866d8e046e0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:58.571456 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-04c52930-465c-4170-acaf-b947260c6ce1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54970c6f86c14078af579866d8e046e0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:58.578741 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 728/1438] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:36:58 2022] DELETE /identity/v3/users/54970c6f86c14078af579866d8e046e0 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:36:58.582999 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e71b31-3b40-4810-86d1-77151c80e523 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1b5e9e9d18d4235973fdc40e44020a7, user_id=0766dfad668d41f189c00cfb38389930, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:58.586182 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e71b31-3b40-4810-86d1-77151c80e523 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:58.604159 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 711/1439] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:36:58 2022] PUT /identity/v3/projects/a1b5e9e9d18d4235973fdc40e44020a7/users/0766dfad668d41f189c00cfb38389930/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:36:58.608356 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c52930-465c-4170-acaf-b947260c6ce1 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:58.610599 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-04c52930-465c-4170-acaf-b947260c6ce1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:58.691015 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c52930-465c-4170-acaf-b947260c6ce1 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:58.779594 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cea2f2d-2a8b-4af8-88ba-6a8ef6a0085c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:58.781586 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea2f2d-2a8b-4af8-88ba-6a8ef6a0085c admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:58.782428 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea2f2d-2a8b-4af8-88ba-6a8ef6a0085c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:58.782701 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea2f2d-2a8b-4af8-88ba-6a8ef6a0085c admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:58.783649 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cea2f2d-2a8b-4af8-88ba-6a8ef6a0085c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:58.785691 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cea2f2d-2a8b-4af8-88ba-6a8ef6a0085c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:58.803458 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 729/1440] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:36:58 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:36:58.805002 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e71b31-3b40-4810-86d1-77151c80e523 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:58.818107 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e71b31-3b40-4810-86d1-77151c80e523 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:58.824862 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0211acb-4005-41a1-8b1f-499a0da7e56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:58.825756 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0211acb-4005-41a1-8b1f-499a0da7e56c admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:58.826137 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0211acb-4005-41a1-8b1f-499a0da7e56c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:58.826503 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0211acb-4005-41a1-8b1f-499a0da7e56c admin admin] PATH_INFO: `/v3/projects/a1b5e9e9d18d4235973fdc40e44020a7/users/0766dfad668d41f189c00cfb38389930/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:58.830582 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0211acb-4005-41a1-8b1f-499a0da7e56c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1b5e9e9d18d4235973fdc40e44020a7, user_id=0766dfad668d41f189c00cfb38389930, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:58.840201 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0211acb-4005-41a1-8b1f-499a0da7e56c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:58.856192 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 712/1441] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:36:58 2022] PUT /identity/v3/projects/a1b5e9e9d18d4235973fdc40e44020a7/users/0766dfad668d41f189c00cfb38389930/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:36:58.867199 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1396429-72ba-4015-ac0d-25fd25038c05 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:58.867481 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1396429-72ba-4015-ac0d-25fd25038c05 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:58.867757 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1396429-72ba-4015-ac0d-25fd25038c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:58.897751 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-b1396429-72ba-4015-ac0d-25fd25038c05 None None] MFA Rules not processed for user `0766dfad668d41f189c00cfb38389930`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:58.898952 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b1396429-72ba-4015-ac0d-25fd25038c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:58.910445 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0211acb-4005-41a1-8b1f-499a0da7e56c admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:58.912037 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e0211acb-4005-41a1-8b1f-499a0da7e56c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:58.920104 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:58.998288 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 730/1442] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:58 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:59.009048 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1396429-72ba-4015-ac0d-25fd25038c05 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:59.010735 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b1396429-72ba-4015-ac0d-25fd25038c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:59.013599 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0211acb-4005-41a1-8b1f-499a0da7e56c admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:59.020047 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667f6908-5626-4b28-9504-def36a07cfa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:59.020047 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f6908-5626-4b28-9504-def36a07cfa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:59.020047 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f6908-5626-4b28-9504-def36a07cfa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:59.025049 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f6908-5626-4b28-9504-def36a07cfa9 admin admin] PATH_INFO: `/v3/projects/7a28f38950ad4721bc14a6a805e1ce9c` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:59.026763 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667f6908-5626-4b28-9504-def36a07cfa9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a28f38950ad4721bc14a6a805e1ce9c)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:59.027465 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:59.029687 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667f6908-5626-4b28-9504-def36a07cfa9 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:59.092377 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1396429-72ba-4015-ac0d-25fd25038c05 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:59.096944 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27602fdb-9080-45cf-ac81-9ab6823190c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:59.097583 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27602fdb-9080-45cf-ac81-9ab6823190c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:59.097863 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27602fdb-9080-45cf-ac81-9ab6823190c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:59.098128 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27602fdb-9080-45cf-ac81-9ab6823190c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:59.129402 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27602fdb-9080-45cf-ac81-9ab6823190c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:59.130711 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27602fdb-9080-45cf-ac81-9ab6823190c8 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:59.146515 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 731/1443] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:36:59 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 09:36:59.262109 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-667f6908-5626-4b28-9504-def36a07cfa9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a28f38950ad4721bc14a6a805e1ce9c', 'request_id': 'req-667f6908-5626-4b28-9504-def36a07cfa9'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:59.282697 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-667f6908-5626-4b28-9504-def36a07cfa9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a28f38950ad4721bc14a6a805e1ce9c', 'request_id': 'req-667f6908-5626-4b28-9504-def36a07cfa9'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:59.290854 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-667f6908-5626-4b28-9504-def36a07cfa9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a28f38950ad4721bc14a6a805e1ce9c', 'request_id': 'req-667f6908-5626-4b28-9504-def36a07cfa9'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:59.305290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-667f6908-5626-4b28-9504-def36a07cfa9 admin admin] The token cache is being invalidate because project 7a28f38950ad4721bc14a6a805e1ce9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:36:59.305660 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-667f6908-5626-4b28-9504-def36a07cfa9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:59.306338 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-667f6908-5626-4b28-9504-def36a07cfa9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:59.307897 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 713/1444] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:36:58 2022] DELETE /identity/v3/projects/7a28f38950ad4721bc14a6a805e1ce9c => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:36:59.312949 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ff0389-1f14-4d5f-a6c8-f16473de63e2 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:59.313294 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ff0389-1f14-4d5f-a6c8-f16473de63e2 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:59.313569 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ff0389-1f14-4d5f-a6c8-f16473de63e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:59.334785 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-c9ff0389-1f14-4d5f-a6c8-f16473de63e2 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:59.335995 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c9ff0389-1f14-4d5f-a6c8-f16473de63e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:59.435619 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 732/1445] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:59.438563 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d27d854-2d66-4af5-87ee-86c7f63a9305 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:59.438851 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d27d854-2d66-4af5-87ee-86c7f63a9305 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:59.439129 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d27d854-2d66-4af5-87ee-86c7f63a9305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:59.461525 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ff0389-1f14-4d5f-a6c8-f16473de63e2 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:59.466207 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c9ff0389-1f14-4d5f-a6c8-f16473de63e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:59.489401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ff0389-1f14-4d5f-a6c8-f16473de63e2 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:59.495624 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-7d27d854-2d66-4af5-87ee-86c7f63a9305 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:59.497663 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7d27d854-2d66-4af5-87ee-86c7f63a9305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:59.502907 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7655ef79-807a-49b1-9f76-d609b8683793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:59.504216 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7655ef79-807a-49b1-9f76-d609b8683793 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:59.504496 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7655ef79-807a-49b1-9f76-d609b8683793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:59.504798 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7655ef79-807a-49b1-9f76-d609b8683793 admin admin] PATH_INFO: `/v3/users/1004f2a0315c46638e027c79a071f9cf` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:59.506890 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7655ef79-807a-49b1-9f76-d609b8683793 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1004f2a0315c46638e027c79a071f9cf)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:59.509050 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7655ef79-807a-49b1-9f76-d609b8683793 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:59.525863 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 714/1446] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:59.532720 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d27d854-2d66-4af5-87ee-86c7f63a9305 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:59.557304 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d27d854-2d66-4af5-87ee-86c7f63a9305 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:59.571002 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406c3409-c56d-4cb0-9d92-fd9656284bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:59.572172 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c3409-c56d-4cb0-9d92-fd9656284bf9 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:59.573450 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c3409-c56d-4cb0-9d92-fd9656284bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:59.573879 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c3409-c56d-4cb0-9d92-fd9656284bf9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:59.575125 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406c3409-c56d-4cb0-9d92-fd9656284bf9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:36:59.575740 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406c3409-c56d-4cb0-9d92-fd9656284bf9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:59.578957 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406c3409-c56d-4cb0-9d92-fd9656284bf9 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:59.599643 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7655ef79-807a-49b1-9f76-d609b8683793 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1004f2a0315c46638e027c79a071f9cf'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:59.600338 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 715/1447] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:36:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:36:59.602164 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406c3409-c56d-4cb0-9d92-fd9656284bf9 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:59.608061 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7655ef79-807a-49b1-9f76-d609b8683793 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1004f2a0315c46638e027c79a071f9cf'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:59.626709 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406c3409-c56d-4cb0-9d92-fd9656284bf9 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:59.629293 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7655ef79-807a-49b1-9f76-d609b8683793 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1004f2a0315c46638e027c79a071f9cf'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:59.633182 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2074f3-eaca-4055-8447-0dd70dfdf06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:59.634503 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2074f3-eaca-4055-8447-0dd70dfdf06d admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:59.635064 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2074f3-eaca-4055-8447-0dd70dfdf06d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:59.635317 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2074f3-eaca-4055-8447-0dd70dfdf06d admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:59.636135 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2074f3-eaca-4055-8447-0dd70dfdf06d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:59.641503 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7655ef79-807a-49b1-9f76-d609b8683793 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1004f2a0315c46638e027c79a071f9cf'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:59.642727 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7655ef79-807a-49b1-9f76-d609b8683793 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1004f2a0315c46638e027c79a071f9cf'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:36:59.651523 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2074f3-eaca-4055-8447-0dd70dfdf06d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:59.666620 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 733/1448] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:36:59 2022] DELETE /identity/v3/users/1004f2a0315c46638e027c79a071f9cf => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:36:59.678131 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13be8b74-19e3-4b7b-b4d5-52b73cf9266a None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:59.678465 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13be8b74-19e3-4b7b-b4d5-52b73cf9266a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:59.678740 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13be8b74-19e3-4b7b-b4d5-52b73cf9266a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:59.727980 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-13be8b74-19e3-4b7b-b4d5-52b73cf9266a None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:36:59.730350 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-13be8b74-19e3-4b7b-b4d5-52b73cf9266a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:59.740344 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 716/1449] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:36:59 2022] POST /identity/v3/projects => generated 405 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:36:59.741620 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2074f3-eaca-4055-8447-0dd70dfdf06d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:59.743907 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5e2074f3-eaca-4055-8447-0dd70dfdf06d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:59.768901 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:59.850041 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 734/1450] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:36:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:36:59.856447 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:36:59.861934 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13be8b74-19e3-4b7b-b4d5-52b73cf9266a None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:59.863378 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-13be8b74-19e3-4b7b-b4d5-52b73cf9266a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:36:59.884846 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2074f3-eaca-4055-8447-0dd70dfdf06d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:36:59.895589 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b011bb-5d40-4534-bcd3-11f843df6f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:36:59.896334 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b011bb-5d40-4534-bcd3-11f843df6f40 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:36:59.896546 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b011bb-5d40-4534-bcd3-11f843df6f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:36:59.897207 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b011bb-5d40-4534-bcd3-11f843df6f40 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:36:59.897918 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b011bb-5d40-4534-bcd3-11f843df6f40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:36:59.908824 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b011bb-5d40-4534-bcd3-11f843df6f40 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:36:59.955452 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 717/1451] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:36:59 2022] POST /identity/v3/users => generated 303 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:36:59.978143 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b011bb-5d40-4534-bcd3-11f843df6f40 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:36:59.996120 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b011bb-5d40-4534-bcd3-11f843df6f40 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:00.003701 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a37b21-2126-434e-9edf-030446ba4744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:00.004340 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a37b21-2126-434e-9edf-030446ba4744 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:00.004861 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a37b21-2126-434e-9edf-030446ba4744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:00.005144 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a37b21-2126-434e-9edf-030446ba4744 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:00.006033 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a37b21-2126-434e-9edf-030446ba4744 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:00.008859 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a37b21-2126-434e-9edf-030446ba4744 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:00.018795 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13be8b74-19e3-4b7b-b4d5-52b73cf9266a None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:00.029357 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 718/1452] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:00.030179 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a37b21-2126-434e-9edf-030446ba4744 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:00.061270 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a37b21-2126-434e-9edf-030446ba4744 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:00.068188 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23378983-f966-40cb-aee3-3b849a71bbd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:00.069211 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23378983-f966-40cb-aee3-3b849a71bbd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:00.069591 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23378983-f966-40cb-aee3-3b849a71bbd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:00.069962 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23378983-f966-40cb-aee3-3b849a71bbd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:00.071584 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23378983-f966-40cb-aee3-3b849a71bbd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:00.073657 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23378983-f966-40cb-aee3-3b849a71bbd1 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:00.087051 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 719/1453] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:00 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:00.091872 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23378983-f966-40cb-aee3-3b849a71bbd1 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:00.116596 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23378983-f966-40cb-aee3-3b849a71bbd1 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:00.130507 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78433fc6-683f-42c8-a5da-c79cbfead702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:00.133936 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78433fc6-683f-42c8-a5da-c79cbfead702 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:00.134344 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78433fc6-683f-42c8-a5da-c79cbfead702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:00.135416 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78433fc6-683f-42c8-a5da-c79cbfead702 admin admin] PATH_INFO: `/v3/projects/79bb883c40684df5b0324fe581362382/users/ea7eac68845648d2817acd9c0ea4c3dd/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:00.152650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6196b076-a70b-442f-b1b8-854396168f6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:00.152650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196b076-a70b-442f-b1b8-854396168f6d service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:00.152650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196b076-a70b-442f-b1b8-854396168f6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:00.152650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196b076-a70b-442f-b1b8-854396168f6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:00.175389 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78433fc6-683f-42c8-a5da-c79cbfead702 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79bb883c40684df5b0324fe581362382, user_id=ea7eac68845648d2817acd9c0ea4c3dd, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:00.178451 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78433fc6-683f-42c8-a5da-c79cbfead702 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:00.193970 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6196b076-a70b-442f-b1b8-854396168f6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:00.198358 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6196b076-a70b-442f-b1b8-854396168f6d service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:00.202222 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 720/1454] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:37:00 2022] PUT /identity/v3/projects/79bb883c40684df5b0324fe581362382/users/ea7eac68845648d2817acd9c0ea4c3dd/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:00.216580 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c449e9d8-dbf1-478c-8cdc-cc3e8a68798e None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:00.220282 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c449e9d8-dbf1-478c-8cdc-cc3e8a68798e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:00.220809 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c449e9d8-dbf1-478c-8cdc-cc3e8a68798e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:00.233833 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 735/1455] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:36:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:37:00.300543 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-c449e9d8-dbf1-478c-8cdc-cc3e8a68798e None None] MFA Rules not processed for user `ea7eac68845648d2817acd9c0ea4c3dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:00.303027 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c449e9d8-dbf1-478c-8cdc-cc3e8a68798e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:00.478755 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 721/1456] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:00 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:37:00.484810 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6196b076-a70b-442f-b1b8-854396168f6d service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:00.503664 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6196b076-a70b-442f-b1b8-854396168f6d service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:00.632098 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc9f9ac-687c-4e39-b231-45b523d45d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:00.632879 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9f9ac-687c-4e39-b231-45b523d45d3a admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:00.633650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9f9ac-687c-4e39-b231-45b523d45d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:00.633960 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9f9ac-687c-4e39-b231-45b523d45d3a admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:00.634685 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc9f9ac-687c-4e39-b231-45b523d45d3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:00.636764 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc9f9ac-687c-4e39-b231-45b523d45d3a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:00.697524 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 736/1457] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:37:00 2022] POST /identity/v3/projects => generated 409 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:37:00.702334 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c449e9d8-dbf1-478c-8cdc-cc3e8a68798e None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:00.723085 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c449e9d8-dbf1-478c-8cdc-cc3e8a68798e None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:00.875497 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc9f9ac-687c-4e39-b231-45b523d45d3a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:00.879105 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ccc9f9ac-687c-4e39-b231-45b523d45d3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:00.918260 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:00.965257 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f95135f-deb7-4a08-9266-ad09749ca3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:00.966542 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f95135f-deb7-4a08-9266-ad09749ca3a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:00.966824 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f95135f-deb7-4a08-9266-ad09749ca3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:00.967084 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f95135f-deb7-4a08-9266-ad09749ca3a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:00.967763 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f95135f-deb7-4a08-9266-ad09749ca3a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:00.969726 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f95135f-deb7-4a08-9266-ad09749ca3a7 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:01.039047 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 722/1458] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:37:00 2022] POST /identity/v3/users => generated 311 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:37:01.039984 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f95135f-deb7-4a08-9266-ad09749ca3a7 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:01.071530 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f95135f-deb7-4a08-9266-ad09749ca3a7 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:01.078685 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0bae88-9090-41d1-a0c0-15c995b483a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:01.079331 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0bae88-9090-41d1-a0c0-15c995b483a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:01.079612 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0bae88-9090-41d1-a0c0-15c995b483a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:01.079882 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0bae88-9090-41d1-a0c0-15c995b483a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:01.081359 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0bae88-9090-41d1-a0c0-15c995b483a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:01.083303 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0bae88-9090-41d1-a0c0-15c995b483a0 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:01.102796 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 723/1459] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:01 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:01.103895 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0bae88-9090-41d1-a0c0-15c995b483a0 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:01.108791 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc9f9ac-687c-4e39-b231-45b523d45d3a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:01.111076 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:01.115837 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0bae88-9090-41d1-a0c0-15c995b483a0 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:01.118068 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d387b47f-e27d-4729-b5ca-339c9c2bb42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:01.118743 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d387b47f-e27d-4729-b5ca-339c9c2bb42a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:01.119007 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d387b47f-e27d-4729-b5ca-339c9c2bb42a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:01.126384 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d387b47f-e27d-4729-b5ca-339c9c2bb42a admin admin] PATH_INFO: `/v3/projects/0ce93fdbf7944edc8f3dd37e95f89022` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:01.127462 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca0321-3d10-4efc-a18d-60fb489b3ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:01.127735 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d387b47f-e27d-4729-b5ca-339c9c2bb42a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ce93fdbf7944edc8f3dd37e95f89022)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:01.128290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca0321-3d10-4efc-a18d-60fb489b3ca2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:01.128678 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca0321-3d10-4efc-a18d-60fb489b3ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:01.129023 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca0321-3d10-4efc-a18d-60fb489b3ca2 admin admin] PATH_INFO: `/v3/projects/0c7f33ee651a41b1a0acdd185bf88c59/users/6ad1d401cf054296babfc2168a7c4e99/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:01.129869 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d387b47f-e27d-4729-b5ca-339c9c2bb42a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:01.147545 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ca0321-3d10-4efc-a18d-60fb489b3ca2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c7f33ee651a41b1a0acdd185bf88c59, user_id=6ad1d401cf054296babfc2168a7c4e99, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:01.151355 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ca0321-3d10-4efc-a18d-60fb489b3ca2 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:01.171401 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 724/1460] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:37:01 2022] PUT /identity/v3/projects/0c7f33ee651a41b1a0acdd185bf88c59/users/6ad1d401cf054296babfc2168a7c4e99/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:01.172669 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca0321-3d10-4efc-a18d-60fb489b3ca2 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:01.186191 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d387b47f-e27d-4729-b5ca-339c9c2bb42a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ce93fdbf7944edc8f3dd37e95f89022', 'request_id': 'req-d387b47f-e27d-4729-b5ca-339c9c2bb42a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:01.194276 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d387b47f-e27d-4729-b5ca-339c9c2bb42a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ce93fdbf7944edc8f3dd37e95f89022', 'request_id': 'req-d387b47f-e27d-4729-b5ca-339c9c2bb42a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:01.204780 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d387b47f-e27d-4729-b5ca-339c9c2bb42a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ce93fdbf7944edc8f3dd37e95f89022', 'request_id': 'req-d387b47f-e27d-4729-b5ca-339c9c2bb42a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:01.218125 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d387b47f-e27d-4729-b5ca-339c9c2bb42a admin admin] The token cache is being invalidate because project 0ce93fdbf7944edc8f3dd37e95f89022 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:37:01.218436 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d387b47f-e27d-4729-b5ca-339c9c2bb42a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:01.219071 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d387b47f-e27d-4729-b5ca-339c9c2bb42a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:01.221025 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 737/1461] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:37:00 2022] DELETE /identity/v3/projects/0ce93fdbf7944edc8f3dd37e95f89022 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:01.225300 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d387b47f-e27d-4729-b5ca-339c9c2bb42a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:01.226708 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d387b47f-e27d-4729-b5ca-339c9c2bb42a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:01.246456 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca0321-3d10-4efc-a18d-60fb489b3ca2 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:01.312136 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e71cea5-647a-4593-a98c-a3d0dc3f6c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:01.312789 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e71cea5-647a-4593-a98c-a3d0dc3f6c6c admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:01.313067 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e71cea5-647a-4593-a98c-a3d0dc3f6c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:01.313305 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d387b47f-e27d-4729-b5ca-339c9c2bb42a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:01.313426 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e71cea5-647a-4593-a98c-a3d0dc3f6c6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:01.313972 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e71cea5-647a-4593-a98c-a3d0dc3f6c6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:01.315906 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e71cea5-647a-4593-a98c-a3d0dc3f6c6c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:01.319318 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f773e393-07fa-4c95-a9ac-e8dac994b799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:01.320087 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f773e393-07fa-4c95-a9ac-e8dac994b799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:01.320367 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f773e393-07fa-4c95-a9ac-e8dac994b799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:01.320679 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f773e393-07fa-4c95-a9ac-e8dac994b799 admin admin] PATH_INFO: `/v3/users/0766dfad668d41f189c00cfb38389930` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:01.322509 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f773e393-07fa-4c95-a9ac-e8dac994b799 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0766dfad668d41f189c00cfb38389930)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:01.324908 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f773e393-07fa-4c95-a9ac-e8dac994b799 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:01.328694 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 725/1462] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:01 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 09:37:01.330366 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e71cea5-647a-4593-a98c-a3d0dc3f6c6c admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:01.332656 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6e71cea5-647a-4593-a98c-a3d0dc3f6c6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:01.346804 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e71cea5-647a-4593-a98c-a3d0dc3f6c6c admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:01.352141 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88291a8-e9af-4ce4-8485-7fc392657fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:01.353496 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88291a8-e9af-4ce4-8485-7fc392657fc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:01.353775 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88291a8-e9af-4ce4-8485-7fc392657fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:01.354104 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88291a8-e9af-4ce4-8485-7fc392657fc8 admin admin] PATH_INFO: `/v3/projects/0c7f33ee651a41b1a0acdd185bf88c59/users/6ad1d401cf054296babfc2168a7c4e99/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:01.357095 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88291a8-e9af-4ce4-8485-7fc392657fc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c7f33ee651a41b1a0acdd185bf88c59, user_id=6ad1d401cf054296babfc2168a7c4e99, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:01.360683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88291a8-e9af-4ce4-8485-7fc392657fc8 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:01.372194 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 726/1463] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:37:01 2022] PUT /identity/v3/projects/0c7f33ee651a41b1a0acdd185bf88c59/users/6ad1d401cf054296babfc2168a7c4e99/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:37:01.384359 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48881ebb-bcd9-4036-b788-a51d79a07170 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:01.384681 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48881ebb-bcd9-4036-b788-a51d79a07170 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:01.384951 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48881ebb-bcd9-4036-b788-a51d79a07170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:01.386051 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:01.388132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f773e393-07fa-4c95-a9ac-e8dac994b799 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0766dfad668d41f189c00cfb38389930'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:01.399001 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f773e393-07fa-4c95-a9ac-e8dac994b799 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0766dfad668d41f189c00cfb38389930'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:01.416542 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-48881ebb-bcd9-4036-b788-a51d79a07170 None None] MFA Rules not processed for user `6ad1d401cf054296babfc2168a7c4e99`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:01.417755 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-48881ebb-bcd9-4036-b788-a51d79a07170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:01.424672 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f773e393-07fa-4c95-a9ac-e8dac994b799 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0766dfad668d41f189c00cfb38389930'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:01.435658 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f773e393-07fa-4c95-a9ac-e8dac994b799 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0766dfad668d41f189c00cfb38389930'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:01.437350 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f773e393-07fa-4c95-a9ac-e8dac994b799 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0766dfad668d41f189c00cfb38389930'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:01.445539 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 738/1464] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:37:01 2022] DELETE /identity/v3/users/0766dfad668d41f189c00cfb38389930 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:37:01.489994 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:01.502684 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 727/1465] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:01 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:01.513446 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f773e393-07fa-4c95-a9ac-e8dac994b799 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:01.515063 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f773e393-07fa-4c95-a9ac-e8dac994b799 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:01.578451 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f773e393-07fa-4c95-a9ac-e8dac994b799 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:01.582387 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dc7485-4757-4fa2-a513-6c4aebec7c08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:01.583075 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc7485-4757-4fa2-a513-6c4aebec7c08 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:01.583424 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc7485-4757-4fa2-a513-6c4aebec7c08 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:01.583760 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc7485-4757-4fa2-a513-6c4aebec7c08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:01.650453 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dc7485-4757-4fa2-a513-6c4aebec7c08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:01.652090 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dc7485-4757-4fa2-a513-6c4aebec7c08 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:01.664198 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 739/1466] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:01 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:37:01.754349 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48881ebb-bcd9-4036-b788-a51d79a07170 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:01.755919 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-48881ebb-bcd9-4036-b788-a51d79a07170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:01.845023 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48881ebb-bcd9-4036-b788-a51d79a07170 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:01.865547 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dc7485-4757-4fa2-a513-6c4aebec7c08 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:01.875359 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dc7485-4757-4fa2-a513-6c4aebec7c08 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:01.880580 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9add6c02-2d54-4022-9e9a-a45e9822b664 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:01.881172 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9add6c02-2d54-4022-9e9a-a45e9822b664 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:01.881451 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9add6c02-2d54-4022-9e9a-a45e9822b664 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:01.881752 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9add6c02-2d54-4022-9e9a-a45e9822b664 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:01.884659 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9add6c02-2d54-4022-9e9a-a45e9822b664 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:01.939171 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a12343-c7e6-4cbc-bdc0-c9ac7e245b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:01.939899 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a12343-c7e6-4cbc-bdc0-c9ac7e245b27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:01.940244 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a12343-c7e6-4cbc-bdc0-c9ac7e245b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:01.940744 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a12343-c7e6-4cbc-bdc0-c9ac7e245b27 admin admin] PATH_INFO: `/v3/projects/a1b5e9e9d18d4235973fdc40e44020a7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:01.941983 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a12343-c7e6-4cbc-bdc0-c9ac7e245b27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1b5e9e9d18d4235973fdc40e44020a7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:01.944700 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a12343-c7e6-4cbc-bdc0-c9ac7e245b27 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:01.967241 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9add6c02-2d54-4022-9e9a-a45e9822b664 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:01.968858 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9add6c02-2d54-4022-9e9a-a45e9822b664 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:01.978488 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 740/1467] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:01 2022] GET /identity/v3/auth/tokens => generated 3282 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 09:37:02.011881 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f2a12343-c7e6-4cbc-bdc0-c9ac7e245b27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1b5e9e9d18d4235973fdc40e44020a7', 'request_id': 'req-f2a12343-c7e6-4cbc-bdc0-c9ac7e245b27'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:02.020740 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f2a12343-c7e6-4cbc-bdc0-c9ac7e245b27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1b5e9e9d18d4235973fdc40e44020a7', 'request_id': 'req-f2a12343-c7e6-4cbc-bdc0-c9ac7e245b27'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:02.028505 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f2a12343-c7e6-4cbc-bdc0-c9ac7e245b27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1b5e9e9d18d4235973fdc40e44020a7', 'request_id': 'req-f2a12343-c7e6-4cbc-bdc0-c9ac7e245b27'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:02.048567 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f2a12343-c7e6-4cbc-bdc0-c9ac7e245b27 admin admin] The token cache is being invalidate because project a1b5e9e9d18d4235973fdc40e44020a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:37:02.048856 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f2a12343-c7e6-4cbc-bdc0-c9ac7e245b27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:02.049661 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f2a12343-c7e6-4cbc-bdc0-c9ac7e245b27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:02.052566 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 728/1468] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:37:01 2022] DELETE /identity/v3/projects/a1b5e9e9d18d4235973fdc40e44020a7 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:02.056791 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa5740d-2001-40fb-8ff1-89dea1bc85fd None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:02.057485 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa5740d-2001-40fb-8ff1-89dea1bc85fd None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:02.057795 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa5740d-2001-40fb-8ff1-89dea1bc85fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:02.094159 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-faa5740d-2001-40fb-8ff1-89dea1bc85fd None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:02.095406 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-faa5740d-2001-40fb-8ff1-89dea1bc85fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:02.119339 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:02.205015 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 741/1469] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:37:02.214386 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00159bce-76e8-4683-82e8-a69a0be4aa1c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:02.214680 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00159bce-76e8-4683-82e8-a69a0be4aa1c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:02.214953 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00159bce-76e8-4683-82e8-a69a0be4aa1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:02.254184 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-00159bce-76e8-4683-82e8-a69a0be4aa1c None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:02.255422 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-00159bce-76e8-4683-82e8-a69a0be4aa1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:02.284597 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 729/1470] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:02.288454 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa5740d-2001-40fb-8ff1-89dea1bc85fd None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:02.309152 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa5740d-2001-40fb-8ff1-89dea1bc85fd None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:02.317153 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ca21e2-7468-4dc6-bb1d-c8e19aa20bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:02.317801 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca21e2-7468-4dc6-bb1d-c8e19aa20bef admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:02.318090 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca21e2-7468-4dc6-bb1d-c8e19aa20bef admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:02.318367 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca21e2-7468-4dc6-bb1d-c8e19aa20bef admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:02.319007 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ca21e2-7468-4dc6-bb1d-c8e19aa20bef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:37:02.319313 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ca21e2-7468-4dc6-bb1d-c8e19aa20bef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:02.320974 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ca21e2-7468-4dc6-bb1d-c8e19aa20bef admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:02.337232 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 742/1471] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:37:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:37:02.338284 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00159bce-76e8-4683-82e8-a69a0be4aa1c None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:02.351816 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00159bce-76e8-4683-82e8-a69a0be4aa1c None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:02.357511 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e172d-958a-4b95-a663-10808e344933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:02.358001 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e172d-958a-4b95-a663-10808e344933 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:02.359205 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e172d-958a-4b95-a663-10808e344933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:02.359490 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e172d-958a-4b95-a663-10808e344933 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:02.360144 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2e172d-958a-4b95-a663-10808e344933 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:02.362110 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2e172d-958a-4b95-a663-10808e344933 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:02.416025 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 730/1472] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:37:02 2022] POST /identity/v3/projects => generated 371 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:37:02.417460 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ca21e2-7468-4dc6-bb1d-c8e19aa20bef admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:02.431003 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ca21e2-7468-4dc6-bb1d-c8e19aa20bef admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:02.506923 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8f3980-0d35-4306-b564-5c6a3884566a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:02.510766 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f3980-0d35-4306-b564-5c6a3884566a admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:02.512539 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f3980-0d35-4306-b564-5c6a3884566a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:02.512894 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f3980-0d35-4306-b564-5c6a3884566a admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:02.514785 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8f3980-0d35-4306-b564-5c6a3884566a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:02.519407 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8f3980-0d35-4306-b564-5c6a3884566a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:02.572090 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 743/1473] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:37:02 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:37:02.573637 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e172d-958a-4b95-a663-10808e344933 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:02.583681 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:02.590834 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e172d-958a-4b95-a663-10808e344933 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:02.599910 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7ac7d3-2c47-44b4-b679-91bd4f668051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:02.600560 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7ac7d3-2c47-44b4-b679-91bd4f668051 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:02.602533 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7ac7d3-2c47-44b4-b679-91bd4f668051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:02.602919 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7ac7d3-2c47-44b4-b679-91bd4f668051 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:02.604241 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7ac7d3-2c47-44b4-b679-91bd4f668051 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:02.610358 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7ac7d3-2c47-44b4-b679-91bd4f668051 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:02.629975 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 731/1474] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 09:37:02.634201 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8f3980-0d35-4306-b564-5c6a3884566a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:02.645448 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8f3980-0d35-4306-b564-5c6a3884566a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:02.650330 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b93956-4f6a-44a7-9635-5c03de8004e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:02.651031 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b93956-4f6a-44a7-9635-5c03de8004e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:02.651401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b93956-4f6a-44a7-9635-5c03de8004e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:02.651761 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b93956-4f6a-44a7-9635-5c03de8004e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:02.659065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b93956-4f6a-44a7-9635-5c03de8004e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:02.660737 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b93956-4f6a-44a7-9635-5c03de8004e9 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:02.676566 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 744/1475] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:02.678634 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7ac7d3-2c47-44b4-b679-91bd4f668051 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:02.692954 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7ac7d3-2c47-44b4-b679-91bd4f668051 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:02.699815 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bf7864-f717-4263-9eb2-a9ee31d269ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:02.699815 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf7864-f717-4263-9eb2-a9ee31d269ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:02.699815 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf7864-f717-4263-9eb2-a9ee31d269ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:02.699815 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf7864-f717-4263-9eb2-a9ee31d269ce admin admin] PATH_INFO: `/v3/projects/3c80c0870a0042518635d61048a92f8b/users/270940138c7e4d8792664916af87be1d/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:02.718339 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bf7864-f717-4263-9eb2-a9ee31d269ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c80c0870a0042518635d61048a92f8b, user_id=270940138c7e4d8792664916af87be1d, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:02.721129 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bf7864-f717-4263-9eb2-a9ee31d269ce admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:02.739839 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 732/1476] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:37:02 2022] PUT /identity/v3/projects/3c80c0870a0042518635d61048a92f8b/users/270940138c7e4d8792664916af87be1d/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:02.752922 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50956deb-4f68-463b-a6b7-2b9cd2ebed84 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:02.753241 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50956deb-4f68-463b-a6b7-2b9cd2ebed84 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:02.753515 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50956deb-4f68-463b-a6b7-2b9cd2ebed84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:02.794324 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-50956deb-4f68-463b-a6b7-2b9cd2ebed84 None None] MFA Rules not processed for user `270940138c7e4d8792664916af87be1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:02.795694 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-50956deb-4f68-463b-a6b7-2b9cd2ebed84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:02.875122 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 745/1477] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:02.879750 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bf7864-f717-4263-9eb2-a9ee31d269ce admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:02.958058 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bf7864-f717-4263-9eb2-a9ee31d269ce admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:02.963114 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda20e6d-1b5f-42aa-93b8-2ebac75f393c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:02.963858 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda20e6d-1b5f-42aa-93b8-2ebac75f393c admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:02.964142 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda20e6d-1b5f-42aa-93b8-2ebac75f393c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:02.964457 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda20e6d-1b5f-42aa-93b8-2ebac75f393c admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:02.965179 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda20e6d-1b5f-42aa-93b8-2ebac75f393c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:02.967050 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda20e6d-1b5f-42aa-93b8-2ebac75f393c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:03.004267 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 733/1478] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:37:02 2022] POST /identity/v3/projects => generated 373 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:37:03.011184 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50956deb-4f68-463b-a6b7-2b9cd2ebed84 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:03.025279 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50956deb-4f68-463b-a6b7-2b9cd2ebed84 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:03.036971 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:03.119805 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ca3d74-267a-4804-8829-a362ed96611a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:03.120508 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ca3d74-267a-4804-8829-a362ed96611a admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:03.120892 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ca3d74-267a-4804-8829-a362ed96611a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:03.121245 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ca3d74-267a-4804-8829-a362ed96611a admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:03.121959 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ca3d74-267a-4804-8829-a362ed96611a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:03.124003 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ca3d74-267a-4804-8829-a362ed96611a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:03.152807 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda20e6d-1b5f-42aa-93b8-2ebac75f393c admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:03.155043 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 746/1479] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:37:03 2022] POST /identity/v3/users => generated 293 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:37:03.164949 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda20e6d-1b5f-42aa-93b8-2ebac75f393c admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:03.170515 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eef4bc-fdb5-4291-909d-184672ce7384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:03.172028 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eef4bc-fdb5-4291-909d-184672ce7384 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:03.172395 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eef4bc-fdb5-4291-909d-184672ce7384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:03.172782 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eef4bc-fdb5-4291-909d-184672ce7384 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:03.173573 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eef4bc-fdb5-4291-909d-184672ce7384 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:03.175266 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eef4bc-fdb5-4291-909d-184672ce7384 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:03.187073 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 734/1480] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 09:37:03.191210 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ca3d74-267a-4804-8829-a362ed96611a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:03.201840 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ca3d74-267a-4804-8829-a362ed96611a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:03.207380 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10764d88-c41c-4a58-b3eb-dad92c7634af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:03.208111 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10764d88-c41c-4a58-b3eb-dad92c7634af admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:03.208401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10764d88-c41c-4a58-b3eb-dad92c7634af admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:03.208715 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10764d88-c41c-4a58-b3eb-dad92c7634af admin admin] PATH_INFO: `/v3/projects/3742d0336c254271818349e89d245803/users/bdaaf1e35ed04a9bab6d7e21b156d36e/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:03.249074 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10764d88-c41c-4a58-b3eb-dad92c7634af admin admin] RBAC: Authorizing `identity:create_grant(project_id=3742d0336c254271818349e89d245803, user_id=bdaaf1e35ed04a9bab6d7e21b156d36e, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:03.255401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10764d88-c41c-4a58-b3eb-dad92c7634af admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:03.292617 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 747/1481] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:37:03 2022] PUT /identity/v3/projects/3742d0336c254271818349e89d245803/users/bdaaf1e35ed04a9bab6d7e21b156d36e/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:03.293904 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eef4bc-fdb5-4291-909d-184672ce7384 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:03.308790 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eef4bc-fdb5-4291-909d-184672ce7384 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:03.454961 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855cd985-e81b-4ea8-886f-b6715605ad05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:03.455632 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855cd985-e81b-4ea8-886f-b6715605ad05 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:03.457221 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855cd985-e81b-4ea8-886f-b6715605ad05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:03.457221 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855cd985-e81b-4ea8-886f-b6715605ad05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:03.458672 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855cd985-e81b-4ea8-886f-b6715605ad05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:03.464864 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855cd985-e81b-4ea8-886f-b6715605ad05 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:03.513021 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 735/1482] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:03.528552 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10764d88-c41c-4a58-b3eb-dad92c7634af admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:03.543823 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10764d88-c41c-4a58-b3eb-dad92c7634af admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:03.597111 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be51bdbf-bd83-47ec-8206-2fd1ce53d7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:03.597873 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51bdbf-bd83-47ec-8206-2fd1ce53d7af admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:03.598124 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51bdbf-bd83-47ec-8206-2fd1ce53d7af admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:03.598406 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51bdbf-bd83-47ec-8206-2fd1ce53d7af admin admin] PATH_INFO: `/v3/projects/3742d0336c254271818349e89d245803/users/bdaaf1e35ed04a9bab6d7e21b156d36e/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:03.601241 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be51bdbf-bd83-47ec-8206-2fd1ce53d7af admin admin] RBAC: Authorizing `identity:create_grant(project_id=3742d0336c254271818349e89d245803, user_id=bdaaf1e35ed04a9bab6d7e21b156d36e, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:03.604022 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be51bdbf-bd83-47ec-8206-2fd1ce53d7af admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:03.626605 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 748/1483] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:37:03 2022] PUT /identity/v3/projects/3742d0336c254271818349e89d245803/users/bdaaf1e35ed04a9bab6d7e21b156d36e/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:03.640337 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0722fa69-31ad-418b-a381-52d95900e489 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:03.640763 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0722fa69-31ad-418b-a381-52d95900e489 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:03.641120 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0722fa69-31ad-418b-a381-52d95900e489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:03.686821 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-0722fa69-31ad-418b-a381-52d95900e489 None None] MFA Rules not processed for user `bdaaf1e35ed04a9bab6d7e21b156d36e`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:03.688072 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0722fa69-31ad-418b-a381-52d95900e489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:03.739401 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:03.821254 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 736/1484] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:03 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:03.883852 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be51bdbf-bd83-47ec-8206-2fd1ce53d7af admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:03.890301 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-be51bdbf-bd83-47ec-8206-2fd1ce53d7af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:03.966516 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be51bdbf-bd83-47ec-8206-2fd1ce53d7af admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:03.970611 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dd8384-e807-438d-954d-ce55bf5b5a17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:03.971253 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd8384-e807-438d-954d-ce55bf5b5a17 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:03.971634 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd8384-e807-438d-954d-ce55bf5b5a17 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:03.971916 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd8384-e807-438d-954d-ce55bf5b5a17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:03.996387 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dd8384-e807-438d-954d-ce55bf5b5a17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:03.997840 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dd8384-e807-438d-954d-ce55bf5b5a17 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:04.008920 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 749/1485] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:03 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:37:04.013426 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0722fa69-31ad-418b-a381-52d95900e489 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:04.015363 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0722fa69-31ad-418b-a381-52d95900e489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:04.114763 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0722fa69-31ad-418b-a381-52d95900e489 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:04.119729 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d78b02-216f-4f30-947f-8512a47c8255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:04.120392 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d78b02-216f-4f30-947f-8512a47c8255 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:04.120699 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d78b02-216f-4f30-947f-8512a47c8255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:04.120974 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d78b02-216f-4f30-947f-8512a47c8255 admin admin] PATH_INFO: `/v3/users/6c295d2a0cf34c268a67e891bb0b709c` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:04.122128 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d78b02-216f-4f30-947f-8512a47c8255 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c295d2a0cf34c268a67e891bb0b709c)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:04.124654 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d78b02-216f-4f30-947f-8512a47c8255 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:04.195492 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-13d78b02-216f-4f30-947f-8512a47c8255 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c295d2a0cf34c268a67e891bb0b709c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:04.210486 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-13d78b02-216f-4f30-947f-8512a47c8255 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c295d2a0cf34c268a67e891bb0b709c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:04.232766 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-13d78b02-216f-4f30-947f-8512a47c8255 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c295d2a0cf34c268a67e891bb0b709c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:04.243509 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-13d78b02-216f-4f30-947f-8512a47c8255 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c295d2a0cf34c268a67e891bb0b709c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:04.244961 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-13d78b02-216f-4f30-947f-8512a47c8255 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c295d2a0cf34c268a67e891bb0b709c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:04.253932 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 737/1486] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:37:04 2022] DELETE /identity/v3/users/6c295d2a0cf34c268a67e891bb0b709c => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:04.258728 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8a9af-b96c-48cb-b4c4-58f43947544c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:04.259209 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8a9af-b96c-48cb-b4c4-58f43947544c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:04.259391 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8a9af-b96c-48cb-b4c4-58f43947544c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:04.310189 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-eea8a9af-b96c-48cb-b4c4-58f43947544c None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:04.313037 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-eea8a9af-b96c-48cb-b4c4-58f43947544c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:04.405027 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 750/1487] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:04.411281 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d78b02-216f-4f30-947f-8512a47c8255 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:04.412907 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-13d78b02-216f-4f30-947f-8512a47c8255 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:04.508598 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:04.538459 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d78b02-216f-4f30-947f-8512a47c8255 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:04.545112 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea8a9af-b96c-48cb-b4c4-58f43947544c None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:04.546924 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe80728-3c4f-4158-9490-c84a12019960 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:04.547528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-eea8a9af-b96c-48cb-b4c4-58f43947544c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:04.547963 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe80728-3c4f-4158-9490-c84a12019960 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:04.548240 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe80728-3c4f-4158-9490-c84a12019960 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:04.548746 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe80728-3c4f-4158-9490-c84a12019960 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:04.554877 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:04.563553 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea8a9af-b96c-48cb-b4c4-58f43947544c None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:04.567048 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe80728-3c4f-4158-9490-c84a12019960 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:04.569704 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe80728-3c4f-4158-9490-c84a12019960 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:04.586043 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522a21f7-57e0-45a5-bf9f-cd029615a440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:04.588112 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a21f7-57e0-45a5-bf9f-cd029615a440 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:04.588727 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a21f7-57e0-45a5-bf9f-cd029615a440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:04.589614 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a21f7-57e0-45a5-bf9f-cd029615a440 admin admin] PATH_INFO: `/v3/users/270940138c7e4d8792664916af87be1d` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:04.594315 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522a21f7-57e0-45a5-bf9f-cd029615a440 admin admin] RBAC: Authorizing `identity:delete_user(user_id=270940138c7e4d8792664916af87be1d)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:04.600022 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 738/1488] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:37:04.600853 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522a21f7-57e0-45a5-bf9f-cd029615a440 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:04.771487 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-522a21f7-57e0-45a5-bf9f-cd029615a440 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '270940138c7e4d8792664916af87be1d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:04.781996 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-522a21f7-57e0-45a5-bf9f-cd029615a440 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '270940138c7e4d8792664916af87be1d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:04.797838 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-522a21f7-57e0-45a5-bf9f-cd029615a440 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '270940138c7e4d8792664916af87be1d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:04.810985 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-522a21f7-57e0-45a5-bf9f-cd029615a440 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '270940138c7e4d8792664916af87be1d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:04.813272 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-522a21f7-57e0-45a5-bf9f-cd029615a440 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '270940138c7e4d8792664916af87be1d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:04.826717 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 751/1489] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:37:04 2022] DELETE /identity/v3/users/270940138c7e4d8792664916af87be1d => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:37:04.832266 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170e8095-3863-424e-af61-e9f5ed50d65c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:04.832934 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170e8095-3863-424e-af61-e9f5ed50d65c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:04.833450 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170e8095-3863-424e-af61-e9f5ed50d65c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:04.861766 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-170e8095-3863-424e-af61-e9f5ed50d65c None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:04.863933 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-170e8095-3863-424e-af61-e9f5ed50d65c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:05.016969 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 739/1490] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:05.047625 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522a21f7-57e0-45a5-bf9f-cd029615a440 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:05.051990 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-522a21f7-57e0-45a5-bf9f-cd029615a440 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:05.245539 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522a21f7-57e0-45a5-bf9f-cd029615a440 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:05.255656 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c1d98-09a6-4f48-9fb0-2cd266c6cff5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:05.256302 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c1d98-09a6-4f48-9fb0-2cd266c6cff5 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:05.256599 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c1d98-09a6-4f48-9fb0-2cd266c6cff5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:05.256869 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c1d98-09a6-4f48-9fb0-2cd266c6cff5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:05.268475 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c1d98-09a6-4f48-9fb0-2cd266c6cff5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:05.270092 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c1d98-09a6-4f48-9fb0-2cd266c6cff5 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:05.284616 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 752/1491] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:37:05.380065 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170e8095-3863-424e-af61-e9f5ed50d65c None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:05.382084 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-170e8095-3863-424e-af61-e9f5ed50d65c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:05.442080 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170e8095-3863-424e-af61-e9f5ed50d65c None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:05.456950 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1fa8b9-0eb1-44f8-b32f-c3af7f515540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:05.457950 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1fa8b9-0eb1-44f8-b32f-c3af7f515540 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:05.458239 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1fa8b9-0eb1-44f8-b32f-c3af7f515540 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:05.458655 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1fa8b9-0eb1-44f8-b32f-c3af7f515540 admin admin] PATH_INFO: `/v3/projects/0c43be6e1ab24eea884e9f3514253828` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:05.460378 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1fa8b9-0eb1-44f8-b32f-c3af7f515540 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c43be6e1ab24eea884e9f3514253828)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:05.463906 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1fa8b9-0eb1-44f8-b32f-c3af7f515540 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:05.584546 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-bf1fa8b9-0eb1-44f8-b32f-c3af7f515540 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c43be6e1ab24eea884e9f3514253828', 'request_id': 'req-bf1fa8b9-0eb1-44f8-b32f-c3af7f515540'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:05.595207 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-bf1fa8b9-0eb1-44f8-b32f-c3af7f515540 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c43be6e1ab24eea884e9f3514253828', 'request_id': 'req-bf1fa8b9-0eb1-44f8-b32f-c3af7f515540'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:05.605163 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-bf1fa8b9-0eb1-44f8-b32f-c3af7f515540 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c43be6e1ab24eea884e9f3514253828', 'request_id': 'req-bf1fa8b9-0eb1-44f8-b32f-c3af7f515540'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:05.625348 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-bf1fa8b9-0eb1-44f8-b32f-c3af7f515540 admin admin] The token cache is being invalidate because project 0c43be6e1ab24eea884e9f3514253828 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:37:05.627059 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-bf1fa8b9-0eb1-44f8-b32f-c3af7f515540 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:05.628224 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-bf1fa8b9-0eb1-44f8-b32f-c3af7f515540 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:05.630895 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 740/1492] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:37:05 2022] DELETE /identity/v3/projects/0c43be6e1ab24eea884e9f3514253828 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:05.634696 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c1d98-09a6-4f48-9fb0-2cd266c6cff5 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:05.646099 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c1d98-09a6-4f48-9fb0-2cd266c6cff5 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:05.740229 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c1dd3-8690-403d-ae5e-abbdef4a6fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:05.740849 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c1dd3-8690-403d-ae5e-abbdef4a6fd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:05.741325 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c1dd3-8690-403d-ae5e-abbdef4a6fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:05.741649 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c1dd3-8690-403d-ae5e-abbdef4a6fd9 admin admin] PATH_INFO: `/v3/users/6f68bbb911c4470dbd085771c806634f` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:05.744203 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c1dd3-8690-403d-ae5e-abbdef4a6fd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f68bbb911c4470dbd085771c806634f)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:05.745062 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c1dd3-8690-403d-ae5e-abbdef4a6fd9 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:05.760707 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1fa8b9-0eb1-44f8-b32f-c3af7f515540 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:05.762242 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-bf1fa8b9-0eb1-44f8-b32f-c3af7f515540 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:05.782250 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1fa8b9-0eb1-44f8-b32f-c3af7f515540 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:05.788717 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:05.796652 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d30e25-9d80-4e84-97ef-0badf0afd55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:05.797445 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d30e25-9d80-4e84-97ef-0badf0afd55f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:05.797741 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d30e25-9d80-4e84-97ef-0badf0afd55f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:05.798284 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d30e25-9d80-4e84-97ef-0badf0afd55f admin admin] PATH_INFO: `/v3/projects/3c80c0870a0042518635d61048a92f8b` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:05.800183 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d30e25-9d80-4e84-97ef-0badf0afd55f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c80c0870a0042518635d61048a92f8b)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:05.802247 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d30e25-9d80-4e84-97ef-0badf0afd55f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:05.847657 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9c8c1dd3-8690-403d-ae5e-abbdef4a6fd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f68bbb911c4470dbd085771c806634f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:05.859137 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9c8c1dd3-8690-403d-ae5e-abbdef4a6fd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f68bbb911c4470dbd085771c806634f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:05.875655 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-19d30e25-9d80-4e84-97ef-0badf0afd55f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c80c0870a0042518635d61048a92f8b', 'request_id': 'req-19d30e25-9d80-4e84-97ef-0badf0afd55f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:05.877039 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9c8c1dd3-8690-403d-ae5e-abbdef4a6fd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f68bbb911c4470dbd085771c806634f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:05.885322 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9c8c1dd3-8690-403d-ae5e-abbdef4a6fd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f68bbb911c4470dbd085771c806634f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:05.886301 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9c8c1dd3-8690-403d-ae5e-abbdef4a6fd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f68bbb911c4470dbd085771c806634f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:05.890903 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-19d30e25-9d80-4e84-97ef-0badf0afd55f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c80c0870a0042518635d61048a92f8b', 'request_id': 'req-19d30e25-9d80-4e84-97ef-0badf0afd55f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:05.893983 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 753/1493] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:37:05 2022] DELETE /identity/v3/users/6f68bbb911c4470dbd085771c806634f => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:05.899367 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-19d30e25-9d80-4e84-97ef-0badf0afd55f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c80c0870a0042518635d61048a92f8b', 'request_id': 'req-19d30e25-9d80-4e84-97ef-0badf0afd55f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:05.902933 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:05.924466 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-19d30e25-9d80-4e84-97ef-0badf0afd55f admin admin] The token cache is being invalidate because project 3c80c0870a0042518635d61048a92f8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:37:05.924466 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-19d30e25-9d80-4e84-97ef-0badf0afd55f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:05.925766 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-19d30e25-9d80-4e84-97ef-0badf0afd55f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:05.928212 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 741/1494] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:37:05 2022] DELETE /identity/v3/projects/3c80c0870a0042518635d61048a92f8b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:05.932457 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c1dd3-8690-403d-ae5e-abbdef4a6fd9 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:05.933954 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9c8c1dd3-8690-403d-ae5e-abbdef4a6fd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:06.171851 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c1dd3-8690-403d-ae5e-abbdef4a6fd9 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:06.178571 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0fb987-e07a-4b2a-90f7-6511b1bffc6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:06.181328 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0fb987-e07a-4b2a-90f7-6511b1bffc6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:06.181822 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0fb987-e07a-4b2a-90f7-6511b1bffc6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:06.182270 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0fb987-e07a-4b2a-90f7-6511b1bffc6a admin admin] PATH_INFO: `/v3/users/bdaaf1e35ed04a9bab6d7e21b156d36e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:06.183892 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0fb987-e07a-4b2a-90f7-6511b1bffc6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdaaf1e35ed04a9bab6d7e21b156d36e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:06.186402 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0fb987-e07a-4b2a-90f7-6511b1bffc6a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:06.262733 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3d0fb987-e07a-4b2a-90f7-6511b1bffc6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdaaf1e35ed04a9bab6d7e21b156d36e'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:06.284276 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3d0fb987-e07a-4b2a-90f7-6511b1bffc6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdaaf1e35ed04a9bab6d7e21b156d36e'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:06.318219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3d0fb987-e07a-4b2a-90f7-6511b1bffc6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdaaf1e35ed04a9bab6d7e21b156d36e'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:06.337815 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3d0fb987-e07a-4b2a-90f7-6511b1bffc6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdaaf1e35ed04a9bab6d7e21b156d36e'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:06.338942 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3d0fb987-e07a-4b2a-90f7-6511b1bffc6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdaaf1e35ed04a9bab6d7e21b156d36e'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:06.348292 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 754/1495] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:37:05 2022] DELETE /identity/v3/users/bdaaf1e35ed04a9bab6d7e21b156d36e => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:06.545866 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d30e25-9d80-4e84-97ef-0badf0afd55f admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:06.548365 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-19d30e25-9d80-4e84-97ef-0badf0afd55f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:06.673625 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d30e25-9d80-4e84-97ef-0badf0afd55f admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:06.791047 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb5b97-9030-48cc-93a1-da410f722560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:06.791685 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb5b97-9030-48cc-93a1-da410f722560 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:06.791963 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb5b97-9030-48cc-93a1-da410f722560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:06.792232 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb5b97-9030-48cc-93a1-da410f722560 admin admin] PATH_INFO: `/v3/projects/8a23c871c2204fdbaeef2619371276f0` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:06.793479 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efb5b97-9030-48cc-93a1-da410f722560 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a23c871c2204fdbaeef2619371276f0)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:06.795555 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efb5b97-9030-48cc-93a1-da410f722560 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:06.854616 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5efb5b97-9030-48cc-93a1-da410f722560 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a23c871c2204fdbaeef2619371276f0', 'request_id': 'req-5efb5b97-9030-48cc-93a1-da410f722560'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:06.861621 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5efb5b97-9030-48cc-93a1-da410f722560 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a23c871c2204fdbaeef2619371276f0', 'request_id': 'req-5efb5b97-9030-48cc-93a1-da410f722560'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:06.867584 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5efb5b97-9030-48cc-93a1-da410f722560 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a23c871c2204fdbaeef2619371276f0', 'request_id': 'req-5efb5b97-9030-48cc-93a1-da410f722560'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:06.881766 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5efb5b97-9030-48cc-93a1-da410f722560 admin admin] The token cache is being invalidate because project 8a23c871c2204fdbaeef2619371276f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:37:06.882171 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5efb5b97-9030-48cc-93a1-da410f722560 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:06.882801 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5efb5b97-9030-48cc-93a1-da410f722560 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:06.884458 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 742/1496] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:37:06 2022] DELETE /identity/v3/projects/8a23c871c2204fdbaeef2619371276f0 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:06.889914 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe6bac-9f90-434f-bc47-1f555396df1a None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:06.890313 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe6bac-9f90-434f-bc47-1f555396df1a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:06.890659 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe6bac-9f90-434f-bc47-1f555396df1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:06.911115 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb5b97-9030-48cc-93a1-da410f722560 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:06.912896 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5efb5b97-9030-48cc-93a1-da410f722560 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:06.914322 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-9bfe6bac-9f90-434f-bc47-1f555396df1a None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:06.915717 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9bfe6bac-9f90-434f-bc47-1f555396df1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:06.928362 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:06.982894 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:07.028339 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb5b97-9030-48cc-93a1-da410f722560 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:07.030236 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 755/1497] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:07.031526 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587983b-e567-4211-9022-d8c2c770c4df None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:07.031815 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587983b-e567-4211-9022-d8c2c770c4df None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:07.032084 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587983b-e567-4211-9022-d8c2c770c4df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:07.040727 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd48650-28d3-4ed8-851a-d239a8290150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:07.042256 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd48650-28d3-4ed8-851a-d239a8290150 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:07.042897 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd48650-28d3-4ed8-851a-d239a8290150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:07.043176 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd48650-28d3-4ed8-851a-d239a8290150 admin admin] PATH_INFO: `/v3/projects/3742d0336c254271818349e89d245803` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:07.045450 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd48650-28d3-4ed8-851a-d239a8290150 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3742d0336c254271818349e89d245803)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:07.047609 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd48650-28d3-4ed8-851a-d239a8290150 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:07.065052 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-3587983b-e567-4211-9022-d8c2c770c4df None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:07.067615 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3587983b-e567-4211-9022-d8c2c770c4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:07.078989 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 756/1498] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:07.083719 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3587983b-e567-4211-9022-d8c2c770c4df None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:07.095165 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3587983b-e567-4211-9022-d8c2c770c4df None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:07.101307 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79eb80-90af-4d06-a4ef-b642840488b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:07.101938 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79eb80-90af-4d06-a4ef-b642840488b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:07.102219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79eb80-90af-4d06-a4ef-b642840488b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:07.102514 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79eb80-90af-4d06-a4ef-b642840488b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:07.104710 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db79eb80-90af-4d06-a4ef-b642840488b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:37:07.104710 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db79eb80-90af-4d06-a4ef-b642840488b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:07.106494 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db79eb80-90af-4d06-a4ef-b642840488b3 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:07.124631 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 757/1499] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:37:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:37:07.125719 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79eb80-90af-4d06-a4ef-b642840488b3 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:07.131101 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-fcd48650-28d3-4ed8-851a-d239a8290150 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3742d0336c254271818349e89d245803', 'request_id': 'req-fcd48650-28d3-4ed8-851a-d239a8290150'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:07.137055 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79eb80-90af-4d06-a4ef-b642840488b3 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:07.139886 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-fcd48650-28d3-4ed8-851a-d239a8290150 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3742d0336c254271818349e89d245803', 'request_id': 'req-fcd48650-28d3-4ed8-851a-d239a8290150'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:07.154661 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-fcd48650-28d3-4ed8-851a-d239a8290150 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3742d0336c254271818349e89d245803', 'request_id': 'req-fcd48650-28d3-4ed8-851a-d239a8290150'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:07.211394 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-fcd48650-28d3-4ed8-851a-d239a8290150 admin admin] The token cache is being invalidate because project 3742d0336c254271818349e89d245803 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:37:07.211694 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-fcd48650-28d3-4ed8-851a-d239a8290150 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:07.212388 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-fcd48650-28d3-4ed8-851a-d239a8290150 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:07.214927 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 743/1500] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:37:06 2022] DELETE /identity/v3/projects/3742d0336c254271818349e89d245803 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:07.220736 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982aa3b-6c8c-46b0-803b-a6cb1dcb13db None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:07.221030 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982aa3b-6c8c-46b0-803b-a6cb1dcb13db None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:07.221308 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982aa3b-6c8c-46b0-803b-a6cb1dcb13db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:07.263286 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-2982aa3b-6c8c-46b0-803b-a6cb1dcb13db None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:07.264567 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2982aa3b-6c8c-46b0-803b-a6cb1dcb13db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:07.264909 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bb88e9-e6cc-442a-aa09-c173992279ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:07.265705 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb88e9-e6cc-442a-aa09-c173992279ba admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:07.266137 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb88e9-e6cc-442a-aa09-c173992279ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:07.266570 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb88e9-e6cc-442a-aa09-c173992279ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:07.267401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bb88e9-e6cc-442a-aa09-c173992279ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:07.278077 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bb88e9-e6cc-442a-aa09-c173992279ba admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:07.286119 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 744/1501] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:07.294149 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e3defc-63fa-4952-b7c8-3c4fca8026ac None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:07.294816 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e3defc-63fa-4952-b7c8-3c4fca8026ac None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:07.295286 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e3defc-63fa-4952-b7c8-3c4fca8026ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:07.319994 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b1e3defc-63fa-4952-b7c8-3c4fca8026ac None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:07.321350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b1e3defc-63fa-4952-b7c8-3c4fca8026ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:07.341461 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 758/1502] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:37:07 2022] POST /identity/v3/projects => generated 401 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:37:07.342365 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 745/1503] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:07.345725 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bb88e9-e6cc-442a-aa09-c173992279ba admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:07.348841 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e3defc-63fa-4952-b7c8-3c4fca8026ac None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:07.349700 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c1bb88e9-e6cc-442a-aa09-c173992279ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:07.364017 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e3defc-63fa-4952-b7c8-3c4fca8026ac None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:07.472061 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bb5c63-36ae-412c-8964-e64b03800e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:07.472804 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb5c63-36ae-412c-8964-e64b03800e4a admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:07.473275 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb5c63-36ae-412c-8964-e64b03800e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:07.473635 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb5c63-36ae-412c-8964-e64b03800e4a admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:07.479395 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bb5c63-36ae-412c-8964-e64b03800e4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:37:07.479743 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bb5c63-36ae-412c-8964-e64b03800e4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:07.481438 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bb5c63-36ae-412c-8964-e64b03800e4a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:07.494996 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 746/1504] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:37:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:37:07.500271 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bb5c63-36ae-412c-8964-e64b03800e4a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:07.501303 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bb88e9-e6cc-442a-aa09-c173992279ba admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:07.506329 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5428025-38cc-4caf-9cfd-071f281e7f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:07.507066 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5428025-38cc-4caf-9cfd-071f281e7f04 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:07.507455 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5428025-38cc-4caf-9cfd-071f281e7f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:07.507828 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5428025-38cc-4caf-9cfd-071f281e7f04 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:07.510956 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bb5c63-36ae-412c-8964-e64b03800e4a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:07.513695 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5428025-38cc-4caf-9cfd-071f281e7f04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:07.516578 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5428025-38cc-4caf-9cfd-071f281e7f04 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:07.526148 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb19cbc-97ac-4a8b-8392-880965a5aeba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:07.526989 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb19cbc-97ac-4a8b-8392-880965a5aeba admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:07.527422 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb19cbc-97ac-4a8b-8392-880965a5aeba admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:07.528381 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb19cbc-97ac-4a8b-8392-880965a5aeba admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:07.529206 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb19cbc-97ac-4a8b-8392-880965a5aeba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:07.531169 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb19cbc-97ac-4a8b-8392-880965a5aeba admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:07.548894 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 759/1505] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:37:07 2022] POST /identity/v3/users => generated 301 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:37:07.551446 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5428025-38cc-4caf-9cfd-071f281e7f04 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:07.579777 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5428025-38cc-4caf-9cfd-071f281e7f04 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:07.598701 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becc0ede-8454-41e0-aaa8-b22d6542eb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:07.600990 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc0ede-8454-41e0-aaa8-b22d6542eb81 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:07.601345 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc0ede-8454-41e0-aaa8-b22d6542eb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:07.601694 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc0ede-8454-41e0-aaa8-b22d6542eb81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:07.602469 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becc0ede-8454-41e0-aaa8-b22d6542eb81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:07.604061 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becc0ede-8454-41e0-aaa8-b22d6542eb81 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:07.606747 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 747/1506] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:37:07 2022] POST /identity/v3/projects => generated 365 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:37:07.611821 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb19cbc-97ac-4a8b-8392-880965a5aeba admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:07.615314 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:07.621756 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 760/1507] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:07.625926 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becc0ede-8454-41e0-aaa8-b22d6542eb81 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:07.639151 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becc0ede-8454-41e0-aaa8-b22d6542eb81 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:07.647062 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:07.726129 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f44d24-ce37-42ad-963d-4cf292fc7962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:07.726740 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f44d24-ce37-42ad-963d-4cf292fc7962 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:07.726942 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f44d24-ce37-42ad-963d-4cf292fc7962 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:07.727336 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f44d24-ce37-42ad-963d-4cf292fc7962 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:07.728303 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f44d24-ce37-42ad-963d-4cf292fc7962 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:07.735278 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f44d24-ce37-42ad-963d-4cf292fc7962 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:07.735900 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb19cbc-97ac-4a8b-8392-880965a5aeba admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:07.745625 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34029a33-3f73-4bce-bc26-aac02defd2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:07.746255 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34029a33-3f73-4bce-bc26-aac02defd2ef admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:07.746588 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34029a33-3f73-4bce-bc26-aac02defd2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:07.746787 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34029a33-3f73-4bce-bc26-aac02defd2ef admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:07.747425 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34029a33-3f73-4bce-bc26-aac02defd2ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:07.749701 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34029a33-3f73-4bce-bc26-aac02defd2ef admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:07.759899 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 761/1508] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 09:37:07.764262 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f44d24-ce37-42ad-963d-4cf292fc7962 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:07.785803 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 748/1509] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:37:07 2022] POST /identity/v3/users => generated 283 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:37:07.789923 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34029a33-3f73-4bce-bc26-aac02defd2ef admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:07.790491 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f44d24-ce37-42ad-963d-4cf292fc7962 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:07.800284 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac05aa4-48a9-43fa-9691-85dfd147e48e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:07.801091 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac05aa4-48a9-43fa-9691-85dfd147e48e admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:07.801374 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac05aa4-48a9-43fa-9691-85dfd147e48e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:07.801652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac05aa4-48a9-43fa-9691-85dfd147e48e admin admin] PATH_INFO: `/v3/projects/e5afc5e3b8e1472bb2a6fe8da9d34e11/users/596c5235b0ca497ba17cf0f1b0a74104/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:07.803325 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34029a33-3f73-4bce-bc26-aac02defd2ef admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:07.814356 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce150903-7321-46a9-9ed5-629604f2459b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:07.815200 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce150903-7321-46a9-9ed5-629604f2459b admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:07.815487 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce150903-7321-46a9-9ed5-629604f2459b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:07.815873 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce150903-7321-46a9-9ed5-629604f2459b admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:07.816406 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce150903-7321-46a9-9ed5-629604f2459b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:07.818173 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce150903-7321-46a9-9ed5-629604f2459b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:07.841156 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac05aa4-48a9-43fa-9691-85dfd147e48e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5afc5e3b8e1472bb2a6fe8da9d34e11, user_id=596c5235b0ca497ba17cf0f1b0a74104, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:07.844944 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac05aa4-48a9-43fa-9691-85dfd147e48e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:07.845482 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 749/1510] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:07.851640 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce150903-7321-46a9-9ed5-629604f2459b admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:07.866260 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce150903-7321-46a9-9ed5-629604f2459b admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:07.872236 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 762/1511] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:37:07 2022] PUT /identity/v3/projects/e5afc5e3b8e1472bb2a6fe8da9d34e11/users/596c5235b0ca497ba17cf0f1b0a74104/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:07.887566 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90adcf66-751d-458a-92d5-59d12780cf24 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:07.887910 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90adcf66-751d-458a-92d5-59d12780cf24 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:07.888190 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90adcf66-751d-458a-92d5-59d12780cf24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:07.956919 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-90adcf66-751d-458a-92d5-59d12780cf24 None None] MFA Rules not processed for user `596c5235b0ca497ba17cf0f1b0a74104`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:07.958152 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-90adcf66-751d-458a-92d5-59d12780cf24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:08.019144 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85982ea1-826a-4736-8922-0dcddf311704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:08.019903 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85982ea1-826a-4736-8922-0dcddf311704 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:08.020259 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85982ea1-826a-4736-8922-0dcddf311704 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:08.020686 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85982ea1-826a-4736-8922-0dcddf311704 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:08.021412 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85982ea1-826a-4736-8922-0dcddf311704 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:08.024182 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85982ea1-826a-4736-8922-0dcddf311704 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:08.037938 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 750/1512] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:08.043066 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85982ea1-826a-4736-8922-0dcddf311704 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:08.059583 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85982ea1-826a-4736-8922-0dcddf311704 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:08.081347 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7734eae0-7a6b-49c9-8fdc-45b66d0d11dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:08.082109 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734eae0-7a6b-49c9-8fdc-45b66d0d11dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:08.082391 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734eae0-7a6b-49c9-8fdc-45b66d0d11dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:08.082672 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734eae0-7a6b-49c9-8fdc-45b66d0d11dd admin admin] PATH_INFO: `/v3/projects/973d5a3560414feb8f6472d9fe26717b/users/c6e87262187140e1a83f586a122905f6/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:08.085896 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 763/1513] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:07 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:08.105773 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90adcf66-751d-458a-92d5-59d12780cf24 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:08.107612 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7734eae0-7a6b-49c9-8fdc-45b66d0d11dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=973d5a3560414feb8f6472d9fe26717b, user_id=c6e87262187140e1a83f586a122905f6, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:08.110430 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7734eae0-7a6b-49c9-8fdc-45b66d0d11dd admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:08.145244 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 751/1514] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:37:08 2022] PUT /identity/v3/projects/973d5a3560414feb8f6472d9fe26717b/users/c6e87262187140e1a83f586a122905f6/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:08.148452 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f2929-48b0-47c3-9cbf-05d505ad4cac None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:08.148768 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f2929-48b0-47c3-9cbf-05d505ad4cac None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:08.149043 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f2929-48b0-47c3-9cbf-05d505ad4cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:08.160874 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90adcf66-751d-458a-92d5-59d12780cf24 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:08.178952 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-df2f2929-48b0-47c3-9cbf-05d505ad4cac None None] MFA Rules not processed for user `45d8920d9a7940abb112dbbb3d7f6e65`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:08.180411 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-df2f2929-48b0-47c3-9cbf-05d505ad4cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:08.261065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0516cb-2187-4f48-8194-33bea2649e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:08.261756 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0516cb-2187-4f48-8194-33bea2649e64 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:08.262046 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0516cb-2187-4f48-8194-33bea2649e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:08.262423 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0516cb-2187-4f48-8194-33bea2649e64 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:08.263097 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 752/1515] 10.222.0.72 () {58 vars in 1015 bytes} [Thu Oct 27 09:37:08 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:08.263681 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0516cb-2187-4f48-8194-33bea2649e64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:08.265649 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7fcef6-969c-476a-a418-8891e337ac27 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:08.266106 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7fcef6-969c-476a-a418-8891e337ac27 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:08.266520 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7fcef6-969c-476a-a418-8891e337ac27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:08.266848 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0516cb-2187-4f48-8194-33bea2649e64 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:08.291849 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:08.315152 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-ad7fcef6-969c-476a-a418-8891e337ac27 None None] MFA Rules not processed for user `c6e87262187140e1a83f586a122905f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:08.316443 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ad7fcef6-969c-476a-a418-8891e337ac27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:08.324074 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 764/1516] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:37:08 2022] POST /identity/v3/projects => generated 401 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:37:08.325378 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0516cb-2187-4f48-8194-33bea2649e64 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:08.328968 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ae0516cb-2187-4f48-8194-33bea2649e64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:08.330085 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:08.422506 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0516cb-2187-4f48-8194-33bea2649e64 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:08.427803 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c0a30a-83c8-48e9-af56-87e2d8001a46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:08.428508 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0a30a-83c8-48e9-af56-87e2d8001a46 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:08.428801 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0a30a-83c8-48e9-af56-87e2d8001a46 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:08.429066 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0a30a-83c8-48e9-af56-87e2d8001a46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:08.437084 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 753/1517] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:08.438433 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7fcef6-969c-476a-a418-8891e337ac27 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:08.440407 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ad7fcef6-969c-476a-a418-8891e337ac27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:08.442224 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c0a30a-83c8-48e9-af56-87e2d8001a46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:08.444254 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c0a30a-83c8-48e9-af56-87e2d8001a46 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:08.454544 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 765/1518] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:37:08.455232 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7fcef6-969c-476a-a418-8891e337ac27 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:08.455532 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c0a30a-83c8-48e9-af56-87e2d8001a46 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:08.459036 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f89969-12f5-48d5-ad70-a7975395acbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:08.459645 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f89969-12f5-48d5-ad70-a7975395acbd service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:08.460103 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f89969-12f5-48d5-ad70-a7975395acbd service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:08.460382 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f89969-12f5-48d5-ad70-a7975395acbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:08.464740 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c0a30a-83c8-48e9-af56-87e2d8001a46 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:08.469753 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f89969-12f5-48d5-ad70-a7975395acbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:08.469929 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f8d0c4-c207-4cf2-80fa-25fa25e7291b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:08.470452 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8d0c4-c207-4cf2-80fa-25fa25e7291b service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:08.470728 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8d0c4-c207-4cf2-80fa-25fa25e7291b service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:08.470995 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8d0c4-c207-4cf2-80fa-25fa25e7291b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:08.471355 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f89969-12f5-48d5-ad70-a7975395acbd service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:08.479379 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f8d0c4-c207-4cf2-80fa-25fa25e7291b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:08.480871 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f8d0c4-c207-4cf2-80fa-25fa25e7291b service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:08.485882 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 754/1519] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:37:08.487170 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f89969-12f5-48d5-ad70-a7975395acbd service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:08.490063 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 766/1520] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:37:08.491153 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f8d0c4-c207-4cf2-80fa-25fa25e7291b service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:08.572242 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f89969-12f5-48d5-ad70-a7975395acbd service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:08.573741 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f8d0c4-c207-4cf2-80fa-25fa25e7291b service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:08.580221 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57af9443-ce50-47ce-aabe-1f716eec6bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:08.581240 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af9443-ce50-47ce-aabe-1f716eec6bbf admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:08.582845 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ecc11b-17f1-40c7-bb59-02746b13d751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:08.582980 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af9443-ce50-47ce-aabe-1f716eec6bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:08.583796 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af9443-ce50-47ce-aabe-1f716eec6bbf admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:08.584181 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ecc11b-17f1-40c7-bb59-02746b13d751 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:08.584690 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ecc11b-17f1-40c7-bb59-02746b13d751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:08.584812 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57af9443-ce50-47ce-aabe-1f716eec6bbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:08.585276 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ecc11b-17f1-40c7-bb59-02746b13d751 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:08.586306 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ecc11b-17f1-40c7-bb59-02746b13d751 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:08.587498 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57af9443-ce50-47ce-aabe-1f716eec6bbf admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:08.588647 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ecc11b-17f1-40c7-bb59-02746b13d751 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:08.641833 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 755/1521] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:37:08 2022] POST /identity/v3/users => generated 307 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:37:08.642647 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 767/1522] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:37:08 2022] POST /identity/v3/projects => generated 365 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:37:08.646684 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ecc11b-17f1-40c7-bb59-02746b13d751 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:08.646809 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57af9443-ce50-47ce-aabe-1f716eec6bbf admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:08.663234 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57af9443-ce50-47ce-aabe-1f716eec6bbf admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:08.744867 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ecc11b-17f1-40c7-bb59-02746b13d751 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:08.753367 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a842d257-4ddc-4c15-b085-a7b1ca680696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:08.754099 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a842d257-4ddc-4c15-b085-a7b1ca680696 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:08.754557 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a842d257-4ddc-4c15-b085-a7b1ca680696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:08.755015 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a842d257-4ddc-4c15-b085-a7b1ca680696 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:08.756308 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a842d257-4ddc-4c15-b085-a7b1ca680696 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:08.759120 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a842d257-4ddc-4c15-b085-a7b1ca680696 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:08.766052 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a9776-eba7-4836-9821-123ae77ed0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:08.766727 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a9776-eba7-4836-9821-123ae77ed0da admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:08.767074 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a9776-eba7-4836-9821-123ae77ed0da admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:08.767350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a9776-eba7-4836-9821-123ae77ed0da admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:08.767987 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8a9776-eba7-4836-9821-123ae77ed0da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:08.770722 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:08.771625 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8a9776-eba7-4836-9821-123ae77ed0da admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:08.788257 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 756/1523] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:08.788710 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 768/1524] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:37:08 2022] POST /identity/v3/users => generated 289 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:37:08.792270 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a842d257-4ddc-4c15-b085-a7b1ca680696 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:08.792406 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a9776-eba7-4836-9821-123ae77ed0da admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:08.805005 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a9776-eba7-4836-9821-123ae77ed0da admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:08.809506 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a842d257-4ddc-4c15-b085-a7b1ca680696 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:08.811831 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2df8976-1536-4f71-98b0-ed38f6dba867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:08.812779 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df8976-1536-4f71-98b0-ed38f6dba867 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:08.813191 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df8976-1536-4f71-98b0-ed38f6dba867 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:08.813883 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df8976-1536-4f71-98b0-ed38f6dba867 admin admin] PATH_INFO: `/v3/projects/93341c892c6b4d8cbe9145446881b8f4/users/9d9284b6a11c4c739851015f449e769a/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:08.818953 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e5477b-5ff2-42af-88f2-2d703d19c2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:08.819673 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e5477b-5ff2-42af-88f2-2d703d19c2dc admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:08.820391 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e5477b-5ff2-42af-88f2-2d703d19c2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:08.820499 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e5477b-5ff2-42af-88f2-2d703d19c2dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:08.821423 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e5477b-5ff2-42af-88f2-2d703d19c2dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:08.822955 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e5477b-5ff2-42af-88f2-2d703d19c2dc admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:08.843009 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2df8976-1536-4f71-98b0-ed38f6dba867 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93341c892c6b4d8cbe9145446881b8f4, user_id=9d9284b6a11c4c739851015f449e769a, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:08.845228 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 769/1525] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:08.849081 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2df8976-1536-4f71-98b0-ed38f6dba867 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:08.849908 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e5477b-5ff2-42af-88f2-2d703d19c2dc admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:08.862692 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:08.864034 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e5477b-5ff2-42af-88f2-2d703d19c2dc admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:08.878844 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 757/1526] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:37:08 2022] PUT /identity/v3/projects/93341c892c6b4d8cbe9145446881b8f4/users/9d9284b6a11c4c739851015f449e769a/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:37:08.882090 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2df8976-1536-4f71-98b0-ed38f6dba867 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:08.978022 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2978adea-ab0f-474c-909a-27b279fa6a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:08.978022 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2978adea-ab0f-474c-909a-27b279fa6a40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:08.978022 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2978adea-ab0f-474c-909a-27b279fa6a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:08.978022 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2978adea-ab0f-474c-909a-27b279fa6a40 admin admin] PATH_INFO: `/v3/projects/f63c1f7fcbed413e9230cb8812f39f16/users/d3e1d7d8c5d340309bf68289ece49965/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:09.001902 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2978adea-ab0f-474c-909a-27b279fa6a40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f63c1f7fcbed413e9230cb8812f39f16, user_id=d3e1d7d8c5d340309bf68289ece49965, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:09.005516 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2978adea-ab0f-474c-909a-27b279fa6a40 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:09.028802 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 770/1527] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:37:08 2022] PUT /identity/v3/projects/f63c1f7fcbed413e9230cb8812f39f16/users/d3e1d7d8c5d340309bf68289ece49965/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:37:09.035587 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2978adea-ab0f-474c-909a-27b279fa6a40 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:09.059723 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2df8976-1536-4f71-98b0-ed38f6dba867 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:09.141643 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2978adea-ab0f-474c-909a-27b279fa6a40 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:09.149019 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6fb92c-cd1e-4da4-8f2e-8245ef52abc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:09.149495 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6fb92c-cd1e-4da4-8f2e-8245ef52abc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:09.149786 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6fb92c-cd1e-4da4-8f2e-8245ef52abc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:09.150149 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6fb92c-cd1e-4da4-8f2e-8245ef52abc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:09.152667 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6fb92c-cd1e-4da4-8f2e-8245ef52abc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:09.156379 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6fb92c-cd1e-4da4-8f2e-8245ef52abc7 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:09.193623 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 771/1528] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:09 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:09.202195 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6fb92c-cd1e-4da4-8f2e-8245ef52abc7 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:09.220985 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3475ca0a-aea2-4673-952b-e7f64d2fcf00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:09.221593 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475ca0a-aea2-4673-952b-e7f64d2fcf00 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:09.223581 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475ca0a-aea2-4673-952b-e7f64d2fcf00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:09.223857 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475ca0a-aea2-4673-952b-e7f64d2fcf00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:09.225244 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3475ca0a-aea2-4673-952b-e7f64d2fcf00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:09.226277 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3475ca0a-aea2-4673-952b-e7f64d2fcf00 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:09.239777 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6fb92c-cd1e-4da4-8f2e-8245ef52abc7 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:09.252128 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46d4f72-e571-4fb6-baa7-ce10e202a251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:09.254528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d4f72-e571-4fb6-baa7-ce10e202a251 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:09.267132 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 758/1529] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:09.270933 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3475ca0a-aea2-4673-952b-e7f64d2fcf00 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:09.278678 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d4f72-e571-4fb6-baa7-ce10e202a251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:09.278975 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d4f72-e571-4fb6-baa7-ce10e202a251 admin admin] PATH_INFO: `/v3/projects/f63c1f7fcbed413e9230cb8812f39f16/users/d3e1d7d8c5d340309bf68289ece49965/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:09.284087 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46d4f72-e571-4fb6-baa7-ce10e202a251 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f63c1f7fcbed413e9230cb8812f39f16, user_id=d3e1d7d8c5d340309bf68289ece49965, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:09.289414 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46d4f72-e571-4fb6-baa7-ce10e202a251 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:09.290775 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3475ca0a-aea2-4673-952b-e7f64d2fcf00 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:09.297770 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b77142a-a360-4b30-822f-bc6e566dcd57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:09.298911 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b77142a-a360-4b30-822f-bc6e566dcd57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:09.299427 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b77142a-a360-4b30-822f-bc6e566dcd57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:09.299938 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b77142a-a360-4b30-822f-bc6e566dcd57 admin admin] PATH_INFO: `/v3/projects/93341c892c6b4d8cbe9145446881b8f4/users/9d9284b6a11c4c739851015f449e769a/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:09.302337 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b77142a-a360-4b30-822f-bc6e566dcd57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93341c892c6b4d8cbe9145446881b8f4, user_id=9d9284b6a11c4c739851015f449e769a, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:09.305624 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b77142a-a360-4b30-822f-bc6e566dcd57 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:09.319196 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 759/1530] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:37:09 2022] PUT /identity/v3/projects/93341c892c6b4d8cbe9145446881b8f4/users/9d9284b6a11c4c739851015f449e769a/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:09.332255 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186dc887-7552-4c6c-b6fd-1ab725a06a2f None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:09.336044 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 772/1531] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:37:09 2022] PUT /identity/v3/projects/f63c1f7fcbed413e9230cb8812f39f16/users/d3e1d7d8c5d340309bf68289ece49965/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:09.337430 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb424b-6235-46a1-890f-42c1ccd2ad26 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:09.337729 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb424b-6235-46a1-890f-42c1ccd2ad26 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:09.338006 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb424b-6235-46a1-890f-42c1ccd2ad26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:09.338359 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186dc887-7552-4c6c-b6fd-1ab725a06a2f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:09.338921 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186dc887-7552-4c6c-b6fd-1ab725a06a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:09.379513 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-4ceb424b-6235-46a1-890f-42c1ccd2ad26 None None] MFA Rules not processed for user `d3e1d7d8c5d340309bf68289ece49965`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:09.380858 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4ceb424b-6235-46a1-890f-42c1ccd2ad26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:09.396538 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-186dc887-7552-4c6c-b6fd-1ab725a06a2f None None] MFA Rules not processed for user `9d9284b6a11c4c739851015f449e769a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:09.401014 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-186dc887-7552-4c6c-b6fd-1ab725a06a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:09.443629 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:09.501452 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 773/1532] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:09 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:09.518018 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb424b-6235-46a1-890f-42c1ccd2ad26 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:09.520493 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4ceb424b-6235-46a1-890f-42c1ccd2ad26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:09.630678 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 760/1533] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:09 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:09.689266 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb424b-6235-46a1-890f-42c1ccd2ad26 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:09.693869 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a051915-f047-4e4e-b90e-e03875b92e55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:09.695612 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a051915-f047-4e4e-b90e-e03875b92e55 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:09.695963 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a051915-f047-4e4e-b90e-e03875b92e55 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:09.696301 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a051915-f047-4e4e-b90e-e03875b92e55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:09.710556 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a051915-f047-4e4e-b90e-e03875b92e55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:09.712568 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a051915-f047-4e4e-b90e-e03875b92e55 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:09.727796 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 774/1534] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:09 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 09:37:09.788393 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186dc887-7552-4c6c-b6fd-1ab725a06a2f None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:09.802087 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186dc887-7552-4c6c-b6fd-1ab725a06a2f None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:09.811065 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9dddbb-1347-41d6-8d67-c6a1a74280ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:09.811749 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9dddbb-1347-41d6-8d67-c6a1a74280ad service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:09.812019 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9dddbb-1347-41d6-8d67-c6a1a74280ad service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:09.812288 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9dddbb-1347-41d6-8d67-c6a1a74280ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:09.840832 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9dddbb-1347-41d6-8d67-c6a1a74280ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:09.843795 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9dddbb-1347-41d6-8d67-c6a1a74280ad service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:09.856946 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:09.877274 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 761/1535] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:09 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:37:09.973733 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a051915-f047-4e4e-b90e-e03875b92e55 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:09.983503 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a051915-f047-4e4e-b90e-e03875b92e55 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:10.006540 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42722b80-21be-421b-9099-1bb2cbaffe4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:10.006540 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42722b80-21be-421b-9099-1bb2cbaffe4f service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:10.006540 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42722b80-21be-421b-9099-1bb2cbaffe4f service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:10.006540 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42722b80-21be-421b-9099-1bb2cbaffe4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:10.073041 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9dddbb-1347-41d6-8d67-c6a1a74280ad service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:10.077686 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-cf9dddbb-1347-41d6-8d67-c6a1a74280ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:10.097518 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9dddbb-1347-41d6-8d67-c6a1a74280ad service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:10.108743 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42722b80-21be-421b-9099-1bb2cbaffe4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:10.109586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f7e040-79a5-43ad-8873-ae73f1beabb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:10.110222 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7e040-79a5-43ad-8873-ae73f1beabb5 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:10.110601 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42722b80-21be-421b-9099-1bb2cbaffe4f service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:10.110724 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7e040-79a5-43ad-8873-ae73f1beabb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:10.110945 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7e040-79a5-43ad-8873-ae73f1beabb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:10.122214 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 775/1536] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:09 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 09:37:10.224552 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f7e040-79a5-43ad-8873-ae73f1beabb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:10.227026 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f7e040-79a5-43ad-8873-ae73f1beabb5 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:10.253160 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 762/1537] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:10 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 09:37:10.823026 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42722b80-21be-421b-9099-1bb2cbaffe4f service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:10.824450 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-42722b80-21be-421b-9099-1bb2cbaffe4f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:10.915289 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42722b80-21be-421b-9099-1bb2cbaffe4f service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:10.923649 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab9320d-1ba3-400a-be5b-44c14bce629e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:10.924466 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9320d-1ba3-400a-be5b-44c14bce629e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:10.924925 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9320d-1ba3-400a-be5b-44c14bce629e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:10.925298 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9320d-1ba3-400a-be5b-44c14bce629e admin admin] PATH_INFO: `/v3/users/ea7eac68845648d2817acd9c0ea4c3dd` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:10.926661 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab9320d-1ba3-400a-be5b-44c14bce629e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea7eac68845648d2817acd9c0ea4c3dd)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:10.931498 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab9320d-1ba3-400a-be5b-44c14bce629e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:10.951373 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:11.004645 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-dab9320d-1ba3-400a-be5b-44c14bce629e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea7eac68845648d2817acd9c0ea4c3dd'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:11.018269 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-dab9320d-1ba3-400a-be5b-44c14bce629e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea7eac68845648d2817acd9c0ea4c3dd'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:11.037297 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-dab9320d-1ba3-400a-be5b-44c14bce629e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea7eac68845648d2817acd9c0ea4c3dd'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:11.051117 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-dab9320d-1ba3-400a-be5b-44c14bce629e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea7eac68845648d2817acd9c0ea4c3dd'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:11.051841 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-dab9320d-1ba3-400a-be5b-44c14bce629e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea7eac68845648d2817acd9c0ea4c3dd'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:11.072243 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 776/1538] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:37:10 2022] DELETE /identity/v3/users/ea7eac68845648d2817acd9c0ea4c3dd => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:11.077139 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad4190-b1e0-4470-981b-0c012f623e41 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:11.077631 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad4190-b1e0-4470-981b-0c012f623e41 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:11.077707 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad4190-b1e0-4470-981b-0c012f623e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:11.101439 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-85ad4190-b1e0-4470-981b-0c012f623e41 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:11.103241 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-85ad4190-b1e0-4470-981b-0c012f623e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:11.145157 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdac1b5-9d00-4286-aa13-49cb1f5fd184 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:11.145157 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdac1b5-9d00-4286-aa13-49cb1f5fd184 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:11.145372 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdac1b5-9d00-4286-aa13-49cb1f5fd184 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:11.147182 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 777/1539] 10.222.0.70 () {54 vars in 921 bytes} [Thu Oct 27 09:37:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:37:11.192176 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61122c1-5b77-46b3-9896-d054fa6f0d28 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:11.192819 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61122c1-5b77-46b3-9896-d054fa6f0d28 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:11.193266 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61122c1-5b77-46b3-9896-d054fa6f0d28 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:11.196145 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 778/1540] 10.222.0.70 () {54 vars in 921 bytes} [Thu Oct 27 09:37:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:37:11.224857 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 763/1541] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:11.238450 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c14a55-7ec5-4b21-a046-39bd5b14c262 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:11.238899 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c14a55-7ec5-4b21-a046-39bd5b14c262 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:11.239362 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c14a55-7ec5-4b21-a046-39bd5b14c262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:11.255792 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ad4190-b1e0-4470-981b-0c012f623e41 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:11.257277 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-85ad4190-b1e0-4470-981b-0c012f623e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:11.261287 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-d3c14a55-7ec5-4b21-a046-39bd5b14c262 None None] MFA Rules not processed for user `dbe4a6e4b6184826b30ac4fc13ad9b1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:11.263833 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d3c14a55-7ec5-4b21-a046-39bd5b14c262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:11.387036 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ad4190-b1e0-4470-981b-0c012f623e41 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:11.395077 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af198e7-9d61-4f65-8417-2ef44455fee0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:11.396172 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af198e7-9d61-4f65-8417-2ef44455fee0 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:11.396690 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af198e7-9d61-4f65-8417-2ef44455fee0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:11.397094 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af198e7-9d61-4f65-8417-2ef44455fee0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:11.440169 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af198e7-9d61-4f65-8417-2ef44455fee0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:11.444695 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 779/1542] 10.222.0.70 () {58 vars in 1018 bytes} [Thu Oct 27 09:37:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:11.450072 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50e256-798d-4830-b378-8a87e8025392 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:11.450405 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af198e7-9d61-4f65-8417-2ef44455fee0 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:11.450801 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50e256-798d-4830-b378-8a87e8025392 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:11.452543 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50e256-798d-4830-b378-8a87e8025392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:11.489758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-7d50e256-798d-4830-b378-8a87e8025392 None None] MFA Rules not processed for user `dbe4a6e4b6184826b30ac4fc13ad9b1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:11.491871 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7d50e256-798d-4830-b378-8a87e8025392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:11.500403 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 764/1543] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 09:37:11.506187 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 780/1544] 10.222.0.70 () {58 vars in 1018 bytes} [Thu Oct 27 09:37:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:12.032020 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af198e7-9d61-4f65-8417-2ef44455fee0 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:12.035964 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8af198e7-9d61-4f65-8417-2ef44455fee0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:12.089061 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af198e7-9d61-4f65-8417-2ef44455fee0 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:12.098511 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b5772e-722b-4f4e-a23a-d77c431c4f8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe4a6e4b6184826b30ac4fc13ad9b1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:12.099168 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b5772e-722b-4f4e-a23a-d77c431c4f8f service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:12.099452 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b5772e-722b-4f4e-a23a-d77c431c4f8f service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:12.099727 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b5772e-722b-4f4e-a23a-d77c431c4f8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:12.118617 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d50e256-798d-4830-b378-8a87e8025392 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:12.120431 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7d50e256-798d-4830-b378-8a87e8025392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:12.132033 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b5772e-722b-4f4e-a23a-d77c431c4f8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:12.133703 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b5772e-722b-4f4e-a23a-d77c431c4f8f service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:12.152555 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d50e256-798d-4830-b378-8a87e8025392 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:12.155337 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 765/1545] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:37:12.163867 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5439b1-8454-425c-abed-c80746fc4f5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe4a6e4b6184826b30ac4fc13ad9b1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:12.165165 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5439b1-8454-425c-abed-c80746fc4f5b service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:12.165446 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5439b1-8454-425c-abed-c80746fc4f5b service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:12.165846 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5439b1-8454-425c-abed-c80746fc4f5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:12.183263 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5439b1-8454-425c-abed-c80746fc4f5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:12.185000 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5439b1-8454-425c-abed-c80746fc4f5b service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:12.199548 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 781/1546] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 09:37:12.664008 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b5772e-722b-4f4e-a23a-d77c431c4f8f service nova] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:12.665544 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-16b5772e-722b-4f4e-a23a-d77c431c4f8f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:12.679440 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b5772e-722b-4f4e-a23a-d77c431c4f8f service nova] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:12.684116 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1e69d9-fb18-4193-ba46-655047fc3152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:12.684783 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e69d9-fb18-4193-ba46-655047fc3152 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:12.685060 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e69d9-fb18-4193-ba46-655047fc3152 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:12.685356 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e69d9-fb18-4193-ba46-655047fc3152 admin admin] PATH_INFO: `/v3/projects/79bb883c40684df5b0324fe581362382` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:12.686502 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1e69d9-fb18-4193-ba46-655047fc3152 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79bb883c40684df5b0324fe581362382)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:12.688650 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1e69d9-fb18-4193-ba46-655047fc3152 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:12.728874 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:12.756397 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3e1e69d9-fb18-4193-ba46-655047fc3152 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79bb883c40684df5b0324fe581362382', 'request_id': 'req-3e1e69d9-fb18-4193-ba46-655047fc3152'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:12.763005 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3e1e69d9-fb18-4193-ba46-655047fc3152 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79bb883c40684df5b0324fe581362382', 'request_id': 'req-3e1e69d9-fb18-4193-ba46-655047fc3152'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:12.773928 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3e1e69d9-fb18-4193-ba46-655047fc3152 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79bb883c40684df5b0324fe581362382', 'request_id': 'req-3e1e69d9-fb18-4193-ba46-655047fc3152'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:12.790957 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3e1e69d9-fb18-4193-ba46-655047fc3152 admin admin] The token cache is being invalidate because project 79bb883c40684df5b0324fe581362382 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:37:12.791330 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3e1e69d9-fb18-4193-ba46-655047fc3152 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:12.792112 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3e1e69d9-fb18-4193-ba46-655047fc3152 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:12.793741 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 766/1547] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:37:12 2022] DELETE /identity/v3/projects/79bb883c40684df5b0324fe581362382 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:12.798190 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5439b1-8454-425c-abed-c80746fc4f5b service nova] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:12.808746 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5439b1-8454-425c-abed-c80746fc4f5b service nova] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:12.865343 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce1502e-0e0b-4005-9746-8bb133cf29d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:12.865965 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1502e-0e0b-4005-9746-8bb133cf29d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:12.866247 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1502e-0e0b-4005-9746-8bb133cf29d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:12.866515 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1502e-0e0b-4005-9746-8bb133cf29d6 admin admin] PATH_INFO: `/v3/users/6ad1d401cf054296babfc2168a7c4e99` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:12.867616 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce1502e-0e0b-4005-9746-8bb133cf29d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ad1d401cf054296babfc2168a7c4e99)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:12.869563 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce1502e-0e0b-4005-9746-8bb133cf29d6 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:12.874441 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:12.915770 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5ce1502e-0e0b-4005-9746-8bb133cf29d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ad1d401cf054296babfc2168a7c4e99'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:12.922540 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5ce1502e-0e0b-4005-9746-8bb133cf29d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ad1d401cf054296babfc2168a7c4e99'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:12.934156 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5ce1502e-0e0b-4005-9746-8bb133cf29d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ad1d401cf054296babfc2168a7c4e99'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:12.941743 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5ce1502e-0e0b-4005-9746-8bb133cf29d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ad1d401cf054296babfc2168a7c4e99'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:12.942351 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5ce1502e-0e0b-4005-9746-8bb133cf29d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ad1d401cf054296babfc2168a7c4e99'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:12.948288 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 782/1548] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:37:12 2022] DELETE /identity/v3/users/6ad1d401cf054296babfc2168a7c4e99 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:13.246694 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1e69d9-fb18-4193-ba46-655047fc3152 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:13.251242 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3e1e69d9-fb18-4193-ba46-655047fc3152 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:13.282312 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1e69d9-fb18-4193-ba46-655047fc3152 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:13.322247 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce1502e-0e0b-4005-9746-8bb133cf29d6 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:13.324268 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5ce1502e-0e0b-4005-9746-8bb133cf29d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:13.377879 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a96654-bd62-4b27-ac92-568e27a3761c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:13.378513 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a96654-bd62-4b27-ac92-568e27a3761c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:13.378787 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a96654-bd62-4b27-ac92-568e27a3761c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:13.379057 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a96654-bd62-4b27-ac92-568e27a3761c admin admin] PATH_INFO: `/v3/projects/0c7f33ee651a41b1a0acdd185bf88c59` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:13.380163 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a96654-bd62-4b27-ac92-568e27a3761c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c7f33ee651a41b1a0acdd185bf88c59)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:13.382116 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a96654-bd62-4b27-ac92-568e27a3761c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:13.399444 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce1502e-0e0b-4005-9746-8bb133cf29d6 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:13.404109 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf29f67a-11da-41e3-90d4-f56428df2f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:13.404881 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf29f67a-11da-41e3-90d4-f56428df2f27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:13.405164 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf29f67a-11da-41e3-90d4-f56428df2f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:13.405441 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf29f67a-11da-41e3-90d4-f56428df2f27 admin admin] PATH_INFO: `/v3/users/c6e87262187140e1a83f586a122905f6` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:13.406597 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf29f67a-11da-41e3-90d4-f56428df2f27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6e87262187140e1a83f586a122905f6)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:13.408532 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf29f67a-11da-41e3-90d4-f56428df2f27 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:13.429046 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e9a96654-bd62-4b27-ac92-568e27a3761c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c7f33ee651a41b1a0acdd185bf88c59', 'request_id': 'req-e9a96654-bd62-4b27-ac92-568e27a3761c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:13.435214 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e9a96654-bd62-4b27-ac92-568e27a3761c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c7f33ee651a41b1a0acdd185bf88c59', 'request_id': 'req-e9a96654-bd62-4b27-ac92-568e27a3761c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:13.442775 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e9a96654-bd62-4b27-ac92-568e27a3761c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c7f33ee651a41b1a0acdd185bf88c59', 'request_id': 'req-e9a96654-bd62-4b27-ac92-568e27a3761c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:13.456468 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e9a96654-bd62-4b27-ac92-568e27a3761c admin admin] The token cache is being invalidate because project 0c7f33ee651a41b1a0acdd185bf88c59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:37:13.456794 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e9a96654-bd62-4b27-ac92-568e27a3761c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:13.457485 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e9a96654-bd62-4b27-ac92-568e27a3761c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:13.459706 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 767/1549] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:37:13 2022] DELETE /identity/v3/projects/0c7f33ee651a41b1a0acdd185bf88c59 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:13.459840 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-cf29f67a-11da-41e3-90d4-f56428df2f27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6e87262187140e1a83f586a122905f6'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:13.464293 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d86b42f-055d-40ef-9cb0-9fd2baa98f06 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:13.464606 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d86b42f-055d-40ef-9cb0-9fd2baa98f06 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:13.465081 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d86b42f-055d-40ef-9cb0-9fd2baa98f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:13.467615 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-cf29f67a-11da-41e3-90d4-f56428df2f27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6e87262187140e1a83f586a122905f6'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:13.480570 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-cf29f67a-11da-41e3-90d4-f56428df2f27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6e87262187140e1a83f586a122905f6'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:13.489381 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-6d86b42f-055d-40ef-9cb0-9fd2baa98f06 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:13.490132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-cf29f67a-11da-41e3-90d4-f56428df2f27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6e87262187140e1a83f586a122905f6'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:13.490535 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6d86b42f-055d-40ef-9cb0-9fd2baa98f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:13.490894 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-cf29f67a-11da-41e3-90d4-f56428df2f27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6e87262187140e1a83f586a122905f6'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:13.497930 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 783/1550] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:37:13 2022] DELETE /identity/v3/users/c6e87262187140e1a83f586a122905f6 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:37:13.503445 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c51a2b-761f-43c0-bf0b-9aaedefeff52 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:13.503729 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c51a2b-761f-43c0-bf0b-9aaedefeff52 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:13.503998 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c51a2b-761f-43c0-bf0b-9aaedefeff52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:13.512830 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:13.524932 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-d7c51a2b-761f-43c0-bf0b-9aaedefeff52 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:13.526636 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d7c51a2b-761f-43c0-bf0b-9aaedefeff52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:13.536235 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:13.564023 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 768/1551] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:13.571749 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a7645-e83e-41d0-8cf1-d2b3f8053856 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:13.572307 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a7645-e83e-41d0-8cf1-d2b3f8053856 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:13.572623 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a7645-e83e-41d0-8cf1-d2b3f8053856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:13.594537 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-7b6a7645-e83e-41d0-8cf1-d2b3f8053856 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:13.595706 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7b6a7645-e83e-41d0-8cf1-d2b3f8053856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:13.614411 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 784/1552] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:13.628564 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c51a2b-761f-43c0-bf0b-9aaedefeff52 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:13.630930 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d7c51a2b-761f-43c0-bf0b-9aaedefeff52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:13.671679 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 769/1553] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:37:13.677193 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6a7645-e83e-41d0-8cf1-d2b3f8053856 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:13.686472 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6a7645-e83e-41d0-8cf1-d2b3f8053856 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:13.691581 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c312319d-896d-4f7d-b0b8-b8497bf385fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:13.692189 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312319d-896d-4f7d-b0b8-b8497bf385fa admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:13.692475 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312319d-896d-4f7d-b0b8-b8497bf385fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:13.692778 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312319d-896d-4f7d-b0b8-b8497bf385fa admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:13.693414 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c312319d-896d-4f7d-b0b8-b8497bf385fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:37:13.693712 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c312319d-896d-4f7d-b0b8-b8497bf385fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:13.695455 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c312319d-896d-4f7d-b0b8-b8497bf385fa admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:13.699954 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c51a2b-761f-43c0-bf0b-9aaedefeff52 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:13.704459 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b22b0b9-fb77-40c8-a35c-82c7b89a7e88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:13.705870 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b22b0b9-fb77-40c8-a35c-82c7b89a7e88 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:13.706311 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b22b0b9-fb77-40c8-a35c-82c7b89a7e88 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:13.706681 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b22b0b9-fb77-40c8-a35c-82c7b89a7e88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:13.708896 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 770/1554] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:37:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:37:13.713202 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c312319d-896d-4f7d-b0b8-b8497bf385fa admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:13.721231 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b22b0b9-fb77-40c8-a35c-82c7b89a7e88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:13.722823 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b22b0b9-fb77-40c8-a35c-82c7b89a7e88 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:13.725289 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c312319d-896d-4f7d-b0b8-b8497bf385fa admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:13.730561 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76408519-f8b0-4cf2-8288-3ebe90ec424f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:13.731239 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76408519-f8b0-4cf2-8288-3ebe90ec424f admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:13.731519 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76408519-f8b0-4cf2-8288-3ebe90ec424f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:13.732036 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76408519-f8b0-4cf2-8288-3ebe90ec424f admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:13.733614 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76408519-f8b0-4cf2-8288-3ebe90ec424f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:13.735724 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76408519-f8b0-4cf2-8288-3ebe90ec424f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:13.736948 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 785/1555] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 09:37:13.766642 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 771/1556] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:37:13 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:37:13.770709 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b22b0b9-fb77-40c8-a35c-82c7b89a7e88 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:13.783556 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b22b0b9-fb77-40c8-a35c-82c7b89a7e88 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:13.882593 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db082408-95e6-4f20-bb97-a890aa50acd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:13.883230 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db082408-95e6-4f20-bb97-a890aa50acd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:13.884380 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db082408-95e6-4f20-bb97-a890aa50acd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:13.884715 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db082408-95e6-4f20-bb97-a890aa50acd6 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:13.885376 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db082408-95e6-4f20-bb97-a890aa50acd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:13.888698 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db082408-95e6-4f20-bb97-a890aa50acd6 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:13.927753 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 786/1557] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:37:13 2022] POST /identity/v3/users => generated 289 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:37:13.931820 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76408519-f8b0-4cf2-8288-3ebe90ec424f admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:13.947151 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76408519-f8b0-4cf2-8288-3ebe90ec424f admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:13.952402 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a476687f-6a62-430d-a0b0-af86ec88b93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:13.953201 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476687f-6a62-430d-a0b0-af86ec88b93b admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:13.953626 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476687f-6a62-430d-a0b0-af86ec88b93b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:13.954083 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476687f-6a62-430d-a0b0-af86ec88b93b admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:13.954728 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a476687f-6a62-430d-a0b0-af86ec88b93b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:13.956877 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a476687f-6a62-430d-a0b0-af86ec88b93b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:13.974054 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 772/1558] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:13.976245 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db082408-95e6-4f20-bb97-a890aa50acd6 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:13.988244 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db082408-95e6-4f20-bb97-a890aa50acd6 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:13.993682 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda18af5-88b8-40c3-8d69-c3e98eaa5089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:13.994317 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda18af5-88b8-40c3-8d69-c3e98eaa5089 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:13.994594 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda18af5-88b8-40c3-8d69-c3e98eaa5089 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:13.994863 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda18af5-88b8-40c3-8d69-c3e98eaa5089 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:13.995535 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda18af5-88b8-40c3-8d69-c3e98eaa5089 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:13.997166 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda18af5-88b8-40c3-8d69-c3e98eaa5089 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:14.013270 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 787/1559] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:14.014687 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a476687f-6a62-430d-a0b0-af86ec88b93b admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:14.025810 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a476687f-6a62-430d-a0b0-af86ec88b93b admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:14.031525 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908e1b4f-66d1-4f93-9ddd-0a17f029e0b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:14.032203 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e1b4f-66d1-4f93-9ddd-0a17f029e0b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:14.032484 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e1b4f-66d1-4f93-9ddd-0a17f029e0b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:14.032801 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e1b4f-66d1-4f93-9ddd-0a17f029e0b8 admin admin] PATH_INFO: `/v3/projects/0019184a911946438dbdae1892ac1a06/users/022d51c1cd8b48d29615ff740209be28/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:14.051569 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908e1b4f-66d1-4f93-9ddd-0a17f029e0b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0019184a911946438dbdae1892ac1a06, user_id=022d51c1cd8b48d29615ff740209be28, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:14.057903 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908e1b4f-66d1-4f93-9ddd-0a17f029e0b8 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:14.063197 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda18af5-88b8-40c3-8d69-c3e98eaa5089 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:14.065130 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-bda18af5-88b8-40c3-8d69-c3e98eaa5089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:14.084779 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 773/1560] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:37:14 2022] PUT /identity/v3/projects/0019184a911946438dbdae1892ac1a06/users/022d51c1cd8b48d29615ff740209be28/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:37:14.090419 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda18af5-88b8-40c3-8d69-c3e98eaa5089 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:14.102736 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784a577-6b3d-414d-b7f3-52f65be57f9c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:14.102972 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784a577-6b3d-414d-b7f3-52f65be57f9c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:14.103338 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784a577-6b3d-414d-b7f3-52f65be57f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:14.114887 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:14.133821 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-d784a577-6b3d-414d-b7f3-52f65be57f9c None None] MFA Rules not processed for user `022d51c1cd8b48d29615ff740209be28`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:14.135011 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d784a577-6b3d-414d-b7f3-52f65be57f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:14.154224 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0fcb1d-df0f-4708-977a-f77317b920c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:14.155842 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0fcb1d-df0f-4708-977a-f77317b920c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:14.156196 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0fcb1d-df0f-4708-977a-f77317b920c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:14.156713 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0fcb1d-df0f-4708-977a-f77317b920c8 admin admin] PATH_INFO: `/v3/projects/973d5a3560414feb8f6472d9fe26717b` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:14.157939 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0fcb1d-df0f-4708-977a-f77317b920c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=973d5a3560414feb8f6472d9fe26717b)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:14.160336 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0fcb1d-df0f-4708-977a-f77317b920c8 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:14.219847 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 774/1561] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:14.225489 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d784a577-6b3d-414d-b7f3-52f65be57f9c None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:14.231669 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:14.241847 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d784a577-6b3d-414d-b7f3-52f65be57f9c None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:14.267868 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3b0fcb1d-df0f-4708-977a-f77317b920c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '973d5a3560414feb8f6472d9fe26717b', 'request_id': 'req-3b0fcb1d-df0f-4708-977a-f77317b920c8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:14.283991 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3b0fcb1d-df0f-4708-977a-f77317b920c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '973d5a3560414feb8f6472d9fe26717b', 'request_id': 'req-3b0fcb1d-df0f-4708-977a-f77317b920c8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:14.296432 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3b0fcb1d-df0f-4708-977a-f77317b920c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '973d5a3560414feb8f6472d9fe26717b', 'request_id': 'req-3b0fcb1d-df0f-4708-977a-f77317b920c8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:14.328782 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3b0fcb1d-df0f-4708-977a-f77317b920c8 admin admin] The token cache is being invalidate because project 973d5a3560414feb8f6472d9fe26717b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:37:14.329442 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3b0fcb1d-df0f-4708-977a-f77317b920c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:14.330093 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3b0fcb1d-df0f-4708-977a-f77317b920c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:14.332535 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 788/1562] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:37:14 2022] DELETE /identity/v3/projects/973d5a3560414feb8f6472d9fe26717b => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:37:14.336561 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0fcb1d-df0f-4708-977a-f77317b920c8 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:14.339184 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3b0fcb1d-df0f-4708-977a-f77317b920c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:14.420449 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164bd85b-db1b-4255-921e-d9889caad8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:14.424396 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164bd85b-db1b-4255-921e-d9889caad8d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:14.424713 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164bd85b-db1b-4255-921e-d9889caad8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:14.425038 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164bd85b-db1b-4255-921e-d9889caad8d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:14.426462 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164bd85b-db1b-4255-921e-d9889caad8d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:14.434214 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164bd85b-db1b-4255-921e-d9889caad8d4 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:14.486883 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0fcb1d-df0f-4708-977a-f77317b920c8 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:14.497137 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2742d05-d72b-4e92-9c36-a303be861dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:14.498016 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2742d05-d72b-4e92-9c36-a303be861dd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:14.498968 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2742d05-d72b-4e92-9c36-a303be861dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:14.499343 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2742d05-d72b-4e92-9c36-a303be861dd5 admin admin] PATH_INFO: `/v3/users/d3e1d7d8c5d340309bf68289ece49965` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:14.506513 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2742d05-d72b-4e92-9c36-a303be861dd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3e1d7d8c5d340309bf68289ece49965)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:14.513618 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 775/1563] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:37:14 2022] POST /identity/v3/projects => generated 377 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:37:14.514769 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2742d05-d72b-4e92-9c36-a303be861dd5 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:14.518358 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164bd85b-db1b-4255-921e-d9889caad8d4 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:14.520575 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-164bd85b-db1b-4255-921e-d9889caad8d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:14.584825 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d2742d05-d72b-4e92-9c36-a303be861dd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3e1d7d8c5d340309bf68289ece49965'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:14.617472 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d2742d05-d72b-4e92-9c36-a303be861dd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3e1d7d8c5d340309bf68289ece49965'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:14.621224 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:14.624467 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164bd85b-db1b-4255-921e-d9889caad8d4 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:14.632679 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c994693b-da80-4ec2-8751-c6f18f7ae701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:14.633465 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c994693b-da80-4ec2-8751-c6f18f7ae701 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:14.633740 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c994693b-da80-4ec2-8751-c6f18f7ae701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:14.634168 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c994693b-da80-4ec2-8751-c6f18f7ae701 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:14.634814 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c994693b-da80-4ec2-8751-c6f18f7ae701 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:14.639591 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c994693b-da80-4ec2-8751-c6f18f7ae701 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:14.655892 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d2742d05-d72b-4e92-9c36-a303be861dd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3e1d7d8c5d340309bf68289ece49965'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:14.675544 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d2742d05-d72b-4e92-9c36-a303be861dd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3e1d7d8c5d340309bf68289ece49965'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:14.675544 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d2742d05-d72b-4e92-9c36-a303be861dd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3e1d7d8c5d340309bf68289ece49965'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:14.689331 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 789/1564] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:37:14 2022] DELETE /identity/v3/users/d3e1d7d8c5d340309bf68289ece49965 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:14.696435 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 776/1565] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:37:14 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:37:14.701126 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2742d05-d72b-4e92-9c36-a303be861dd5 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:14.703469 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d2742d05-d72b-4e92-9c36-a303be861dd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:14.802143 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2742d05-d72b-4e92-9c36-a303be861dd5 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:14.807768 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af437e1-46f8-404e-ae9b-e4f1061a36f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:14.808642 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af437e1-46f8-404e-ae9b-e4f1061a36f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:14.809016 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af437e1-46f8-404e-ae9b-e4f1061a36f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:14.809406 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af437e1-46f8-404e-ae9b-e4f1061a36f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:14.810224 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af437e1-46f8-404e-ae9b-e4f1061a36f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:14.812792 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af437e1-46f8-404e-ae9b-e4f1061a36f1 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:14.828274 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 790/1566] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:14 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 09:37:14.829784 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c994693b-da80-4ec2-8751-c6f18f7ae701 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:14.846568 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c994693b-da80-4ec2-8751-c6f18f7ae701 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:14.853848 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef944fc-afa9-4488-bdee-b542deb77d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:14.854621 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef944fc-afa9-4488-bdee-b542deb77d02 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:14.855125 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef944fc-afa9-4488-bdee-b542deb77d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:14.855495 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef944fc-afa9-4488-bdee-b542deb77d02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:14.857001 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef944fc-afa9-4488-bdee-b542deb77d02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:14.859248 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef944fc-afa9-4488-bdee-b542deb77d02 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:14.875622 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 777/1567] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:14 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:14.879633 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af437e1-46f8-404e-ae9b-e4f1061a36f1 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:14.922262 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af437e1-46f8-404e-ae9b-e4f1061a36f1 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:14.931034 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa675867-a45f-440f-98a3-9b780de197bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:14.931034 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa675867-a45f-440f-98a3-9b780de197bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:14.931491 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa675867-a45f-440f-98a3-9b780de197bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:14.931866 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa675867-a45f-440f-98a3-9b780de197bb admin admin] PATH_INFO: `/v3/projects/317401fd1d67455d9a80b629a92de188/users/47b37b5864aa4595905ac7fc373b790c/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:14.963534 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa675867-a45f-440f-98a3-9b780de197bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=317401fd1d67455d9a80b629a92de188, user_id=47b37b5864aa4595905ac7fc373b790c, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:14.966820 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa675867-a45f-440f-98a3-9b780de197bb admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:15.001527 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 791/1568] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:37:14 2022] PUT /identity/v3/projects/317401fd1d67455d9a80b629a92de188/users/47b37b5864aa4595905ac7fc373b790c/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:15.001685 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd298827-de5e-445c-afca-3f30db8a8093 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:15.002077 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd298827-de5e-445c-afca-3f30db8a8093 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:15.002621 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd298827-de5e-445c-afca-3f30db8a8093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:15.069049 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-fd298827-de5e-445c-afca-3f30db8a8093 None None] MFA Rules not processed for user `47b37b5864aa4595905ac7fc373b790c`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:15.070570 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fd298827-de5e-445c-afca-3f30db8a8093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:15.130819 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa675867-a45f-440f-98a3-9b780de197bb admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:15.132644 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-aa675867-a45f-440f-98a3-9b780de197bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:15.183291 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:15.198783 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 778/1569] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:15.199814 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd298827-de5e-445c-afca-3f30db8a8093 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:15.260812 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa675867-a45f-440f-98a3-9b780de197bb admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:15.267019 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ef1511-f6f3-4a6c-be35-8eb329809ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:15.267769 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef1511-f6f3-4a6c-be35-8eb329809ef3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:15.276354 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd298827-de5e-445c-afca-3f30db8a8093 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:15.280039 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef1511-f6f3-4a6c-be35-8eb329809ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:15.280710 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef1511-f6f3-4a6c-be35-8eb329809ef3 admin admin] PATH_INFO: `/v3/users/596c5235b0ca497ba17cf0f1b0a74104` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:15.284123 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ef1511-f6f3-4a6c-be35-8eb329809ef3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=596c5235b0ca497ba17cf0f1b0a74104)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:15.286396 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ef1511-f6f3-4a6c-be35-8eb329809ef3 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:15.286992 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0cff4e-c3f2-41b9-98aa-27214d3f73d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:15.288473 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0cff4e-c3f2-41b9-98aa-27214d3f73d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:15.288825 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0cff4e-c3f2-41b9-98aa-27214d3f73d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:15.289219 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0cff4e-c3f2-41b9-98aa-27214d3f73d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:15.289721 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0cff4e-c3f2-41b9-98aa-27214d3f73d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:15.291845 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0cff4e-c3f2-41b9-98aa-27214d3f73d1 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:15.339656 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 779/1570] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:37:15 2022] POST /identity/v3/projects => generated 377 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:37:15.341208 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0cff4e-c3f2-41b9-98aa-27214d3f73d1 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:15.374606 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-42ef1511-f6f3-4a6c-be35-8eb329809ef3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '596c5235b0ca497ba17cf0f1b0a74104'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:15.376311 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:15.392138 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-42ef1511-f6f3-4a6c-be35-8eb329809ef3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '596c5235b0ca497ba17cf0f1b0a74104'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:15.410750 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-42ef1511-f6f3-4a6c-be35-8eb329809ef3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '596c5235b0ca497ba17cf0f1b0a74104'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:15.417574 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0cff4e-c3f2-41b9-98aa-27214d3f73d1 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:15.424773 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea799aee-dec1-4dee-b760-195405d3d7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:15.425376 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea799aee-dec1-4dee-b760-195405d3d7c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:15.425651 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea799aee-dec1-4dee-b760-195405d3d7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:15.425917 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea799aee-dec1-4dee-b760-195405d3d7c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:15.426650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-42ef1511-f6f3-4a6c-be35-8eb329809ef3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '596c5235b0ca497ba17cf0f1b0a74104'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:15.426762 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea799aee-dec1-4dee-b760-195405d3d7c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:15.427471 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-42ef1511-f6f3-4a6c-be35-8eb329809ef3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '596c5235b0ca497ba17cf0f1b0a74104'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:15.428807 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea799aee-dec1-4dee-b760-195405d3d7c4 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:15.437765 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 792/1571] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:37:15 2022] DELETE /identity/v3/users/596c5235b0ca497ba17cf0f1b0a74104 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:15.442583 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f63bb1-cc00-434b-95cf-fd9f8797c12f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:15.442920 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f63bb1-cc00-434b-95cf-fd9f8797c12f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:15.443350 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f63bb1-cc00-434b-95cf-fd9f8797c12f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:15.459592 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 780/1572] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:37:15 2022] POST /identity/v3/users => generated 295 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:37:15.464696 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea799aee-dec1-4dee-b760-195405d3d7c4 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:15.471599 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ea799aee-dec1-4dee-b760-195405d3d7c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:15.486262 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-45f63bb1-cc00-434b-95cf-fd9f8797c12f None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:15.487610 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-45f63bb1-cc00-434b-95cf-fd9f8797c12f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:15.494897 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea799aee-dec1-4dee-b760-195405d3d7c4 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:15.584101 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1594d-abe9-494c-880d-cfef42380518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:15.584824 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1594d-abe9-494c-880d-cfef42380518 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:15.585108 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1594d-abe9-494c-880d-cfef42380518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:15.585519 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1594d-abe9-494c-880d-cfef42380518 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:15.586332 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd1594d-abe9-494c-880d-cfef42380518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:15.588325 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd1594d-abe9-494c-880d-cfef42380518 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:15.599474 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 781/1573] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:15.603551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1594d-abe9-494c-880d-cfef42380518 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:15.614604 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1594d-abe9-494c-880d-cfef42380518 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:15.620652 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3dda7-0b81-4af0-8342-57f1853b3e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:15.621311 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3dda7-0b81-4af0-8342-57f1853b3e48 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:15.621587 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3dda7-0b81-4af0-8342-57f1853b3e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:15.621851 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3dda7-0b81-4af0-8342-57f1853b3e48 admin admin] PATH_INFO: `/v3/projects/9ca3d22f212346daa6c39ea6ef03d553/users/6f8ae5f9e6274665be8f6cf17c740115/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:15.624377 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:15.628451 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 793/1574] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:37:15.632093 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f63bb1-cc00-434b-95cf-fd9f8797c12f None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:15.632093 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-45f63bb1-cc00-434b-95cf-fd9f8797c12f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:15.644093 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a3dda7-0b81-4af0-8342-57f1853b3e48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ca3d22f212346daa6c39ea6ef03d553, user_id=6f8ae5f9e6274665be8f6cf17c740115, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:15.644294 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f63bb1-cc00-434b-95cf-fd9f8797c12f None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:15.646765 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a3dda7-0b81-4af0-8342-57f1853b3e48 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:15.649493 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3172014-9bf9-4219-86d0-e8d9394ca04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:15.650367 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3172014-9bf9-4219-86d0-e8d9394ca04c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:15.650754 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3172014-9bf9-4219-86d0-e8d9394ca04c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:15.651114 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3172014-9bf9-4219-86d0-e8d9394ca04c admin admin] PATH_INFO: `/v3/projects/f63c1f7fcbed413e9230cb8812f39f16` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:15.652698 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3172014-9bf9-4219-86d0-e8d9394ca04c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f63c1f7fcbed413e9230cb8812f39f16)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:15.655468 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3172014-9bf9-4219-86d0-e8d9394ca04c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:15.672010 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 782/1575] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:37:15 2022] PUT /identity/v3/projects/9ca3d22f212346daa6c39ea6ef03d553/users/6f8ae5f9e6274665be8f6cf17c740115/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:37:15.673380 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3dda7-0b81-4af0-8342-57f1853b3e48 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:15.675085 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-22a3dda7-0b81-4af0-8342-57f1853b3e48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:15.714805 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-c3172014-9bf9-4219-86d0-e8d9394ca04c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f63c1f7fcbed413e9230cb8812f39f16', 'request_id': 'req-c3172014-9bf9-4219-86d0-e8d9394ca04c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:15.724557 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-c3172014-9bf9-4219-86d0-e8d9394ca04c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f63c1f7fcbed413e9230cb8812f39f16', 'request_id': 'req-c3172014-9bf9-4219-86d0-e8d9394ca04c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:15.732107 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-c3172014-9bf9-4219-86d0-e8d9394ca04c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f63c1f7fcbed413e9230cb8812f39f16', 'request_id': 'req-c3172014-9bf9-4219-86d0-e8d9394ca04c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:15.734484 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3dda7-0b81-4af0-8342-57f1853b3e48 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:15.743387 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-c3172014-9bf9-4219-86d0-e8d9394ca04c admin admin] The token cache is being invalidate because project f63c1f7fcbed413e9230cb8812f39f16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:37:15.743671 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-c3172014-9bf9-4219-86d0-e8d9394ca04c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:15.744312 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-c3172014-9bf9-4219-86d0-e8d9394ca04c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:15.746053 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 794/1576] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:37:15 2022] DELETE /identity/v3/projects/f63c1f7fcbed413e9230cb8812f39f16 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:15.746607 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3172014-9bf9-4219-86d0-e8d9394ca04c admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:15.747866 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c3172014-9bf9-4219-86d0-e8d9394ca04c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:15.799385 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dad2f3b-d018-4850-bdef-bc48552b628b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:15.800331 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad2f3b-d018-4850-bdef-bc48552b628b service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:15.800661 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad2f3b-d018-4850-bdef-bc48552b628b service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:15.800932 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad2f3b-d018-4850-bdef-bc48552b628b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:15.803059 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9dad2f3b-d018-4850-bdef-bc48552b628b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:15.807784 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:15.827665 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3172014-9bf9-4219-86d0-e8d9394ca04c admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:15.833188 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740d0c09-9c78-4e2c-925e-1f591e887fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:15.833905 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740d0c09-9c78-4e2c-925e-1f591e887fee admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:15.834239 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740d0c09-9c78-4e2c-925e-1f591e887fee admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:15.834573 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740d0c09-9c78-4e2c-925e-1f591e887fee admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:15.835280 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740d0c09-9c78-4e2c-925e-1f591e887fee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:15.836981 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740d0c09-9c78-4e2c-925e-1f591e887fee admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:15.852107 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 795/1577] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:15.852107 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7aebe1-5e4d-4575-b48b-0ec2ec6a0e17 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:15.852107 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7aebe1-5e4d-4575-b48b-0ec2ec6a0e17 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:15.852107 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7aebe1-5e4d-4575-b48b-0ec2ec6a0e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:15.871385 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-1f7aebe1-5e4d-4575-b48b-0ec2ec6a0e17 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:15.872602 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1f7aebe1-5e4d-4575-b48b-0ec2ec6a0e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:15.882690 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 796/1578] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:15.883542 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7aebe1-5e4d-4575-b48b-0ec2ec6a0e17 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:15.885723 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dad2f3b-d018-4850-bdef-bc48552b628b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:15.887179 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dad2f3b-d018-4850-bdef-bc48552b628b service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:15.893905 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7aebe1-5e4d-4575-b48b-0ec2ec6a0e17 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:15.898585 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c71fb2f-e32c-4a4e-bc0e-a44c7289c3e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:15.899779 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71fb2f-e32c-4a4e-bc0e-a44c7289c3e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:15.899914 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 783/1579] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:37:15.900022 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71fb2f-e32c-4a4e-bc0e-a44c7289c3e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:15.900022 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71fb2f-e32c-4a4e-bc0e-a44c7289c3e4 admin admin] PATH_INFO: `/v3/projects/9ca3d22f212346daa6c39ea6ef03d553/users/6f8ae5f9e6274665be8f6cf17c740115/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:15.901038 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af49c49-1c9a-4526-a867-b0c6b2a06f7e None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:15.901500 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af49c49-1c9a-4526-a867-b0c6b2a06f7e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:15.901867 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af49c49-1c9a-4526-a867-b0c6b2a06f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:15.902149 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c71fb2f-e32c-4a4e-bc0e-a44c7289c3e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ca3d22f212346daa6c39ea6ef03d553, user_id=6f8ae5f9e6274665be8f6cf17c740115, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:15.904862 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c71fb2f-e32c-4a4e-bc0e-a44c7289c3e4 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:15.916031 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 797/1580] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:37:15 2022] PUT /identity/v3/projects/9ca3d22f212346daa6c39ea6ef03d553/users/6f8ae5f9e6274665be8f6cf17c740115/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:15.926449 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b230f-c8d8-4e39-b6a0-47ee856abf2c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:15.926855 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b230f-c8d8-4e39-b6a0-47ee856abf2c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:15.926964 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-5af49c49-1c9a-4526-a867-b0c6b2a06f7e None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:15.927050 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b230f-c8d8-4e39-b6a0-47ee856abf2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:15.928533 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5af49c49-1c9a-4526-a867-b0c6b2a06f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:15.960653 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-c08b230f-c8d8-4e39-b6a0-47ee856abf2c None None] MFA Rules not processed for user `6f8ae5f9e6274665be8f6cf17c740115`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:15.962015 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c08b230f-c8d8-4e39-b6a0-47ee856abf2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:16.005975 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:16.031793 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 784/1581] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:16.036648 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af49c49-1c9a-4526-a867-b0c6b2a06f7e None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:16.051889 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af49c49-1c9a-4526-a867-b0c6b2a06f7e None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:16.055765 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 798/1582] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:15 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:16.063485 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937bce71-5fcf-407c-b245-e91c4cc95b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:16.064274 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937bce71-5fcf-407c-b245-e91c4cc95b5f admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:16.064608 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937bce71-5fcf-407c-b245-e91c4cc95b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:16.064889 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937bce71-5fcf-407c-b245-e91c4cc95b5f admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:16.065544 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937bce71-5fcf-407c-b245-e91c4cc95b5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:37:16.065846 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937bce71-5fcf-407c-b245-e91c4cc95b5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:16.067806 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937bce71-5fcf-407c-b245-e91c4cc95b5f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:16.086009 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 785/1583] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:37:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:37:16.088662 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08b230f-c8d8-4e39-b6a0-47ee856abf2c None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:16.089379 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937bce71-5fcf-407c-b245-e91c4cc95b5f admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:16.092361 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c08b230f-c8d8-4e39-b6a0-47ee856abf2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:16.105870 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937bce71-5fcf-407c-b245-e91c4cc95b5f admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:16.112366 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd6e197-0951-420a-bc07-5b72fab1b4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:16.113351 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd6e197-0951-420a-bc07-5b72fab1b4c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:16.113840 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd6e197-0951-420a-bc07-5b72fab1b4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:16.114301 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd6e197-0951-420a-bc07-5b72fab1b4c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:16.115286 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd6e197-0951-420a-bc07-5b72fab1b4c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:16.118520 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd6e197-0951-420a-bc07-5b72fab1b4c4 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:16.181659 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 786/1584] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:37:16 2022] POST /identity/v3/projects => generated 365 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:37:16.181826 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08b230f-c8d8-4e39-b6a0-47ee856abf2c None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:16.183276 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd6e197-0951-420a-bc07-5b72fab1b4c4 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:16.321251 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398ef161-330c-4aab-88e7-e143c52a8bf5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:16.322267 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ef161-330c-4aab-88e7-e143c52a8bf5 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:16.322618 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ef161-330c-4aab-88e7-e143c52a8bf5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:16.325329 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ef161-330c-4aab-88e7-e143c52a8bf5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:16.355433 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd6e197-0951-420a-bc07-5b72fab1b4c4 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:16.362344 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:16.377478 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b67011d-4faa-4244-91ec-d210e7ce08c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:16.378111 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b67011d-4faa-4244-91ec-d210e7ce08c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:16.378393 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b67011d-4faa-4244-91ec-d210e7ce08c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:16.380966 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b67011d-4faa-4244-91ec-d210e7ce08c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:16.383830 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b67011d-4faa-4244-91ec-d210e7ce08c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:16.389743 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b67011d-4faa-4244-91ec-d210e7ce08c7 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:16.446485 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 787/1585] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:37:16 2022] POST /identity/v3/users => generated 283 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:37:16.447971 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b67011d-4faa-4244-91ec-d210e7ce08c7 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:16.448089 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398ef161-330c-4aab-88e7-e143c52a8bf5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:16.450213 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398ef161-330c-4aab-88e7-e143c52a8bf5 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:16.453441 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7b67011d-4faa-4244-91ec-d210e7ce08c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:16.465721 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 799/1586] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:16 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:37:16.466352 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398ef161-330c-4aab-88e7-e143c52a8bf5 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:16.480860 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b67011d-4faa-4244-91ec-d210e7ce08c7 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:16.487663 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398ef161-330c-4aab-88e7-e143c52a8bf5 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:16.493737 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac96fe7-65d1-4137-9497-7fbca30f7a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:16.497668 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac96fe7-65d1-4137-9497-7fbca30f7a09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:16.498854 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac96fe7-65d1-4137-9497-7fbca30f7a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:16.499413 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac96fe7-65d1-4137-9497-7fbca30f7a09 admin admin] PATH_INFO: `/v3/projects/e5afc5e3b8e1472bb2a6fe8da9d34e11` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:16.500954 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb19fb7-fbf4-443c-88d5-7c363f68cef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:16.502104 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb19fb7-fbf4-443c-88d5-7c363f68cef9 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:16.502104 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb19fb7-fbf4-443c-88d5-7c363f68cef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:16.503097 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac96fe7-65d1-4137-9497-7fbca30f7a09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5afc5e3b8e1472bb2a6fe8da9d34e11)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:16.504695 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb19fb7-fbf4-443c-88d5-7c363f68cef9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:16.505339 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb19fb7-fbf4-443c-88d5-7c363f68cef9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:16.507799 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac96fe7-65d1-4137-9497-7fbca30f7a09 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:16.509113 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb19fb7-fbf4-443c-88d5-7c363f68cef9 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:16.536618 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 800/1587] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:16 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:16.537643 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb19fb7-fbf4-443c-88d5-7c363f68cef9 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:16.550931 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb19fb7-fbf4-443c-88d5-7c363f68cef9 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:16.557024 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b0c3b-5446-4390-9eb9-f80bb22e9e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:16.557743 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b0c3b-5446-4390-9eb9-f80bb22e9e80 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:16.558025 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b0c3b-5446-4390-9eb9-f80bb22e9e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:16.558441 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b0c3b-5446-4390-9eb9-f80bb22e9e80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:16.559086 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7b0c3b-5446-4390-9eb9-f80bb22e9e80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:16.560748 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7b0c3b-5446-4390-9eb9-f80bb22e9e80 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:16.586308 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 801/1588] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:16 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:16.586308 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b0c3b-5446-4390-9eb9-f80bb22e9e80 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:16.609788 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b0c3b-5446-4390-9eb9-f80bb22e9e80 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:16.625483 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-1ac96fe7-65d1-4137-9497-7fbca30f7a09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5afc5e3b8e1472bb2a6fe8da9d34e11', 'request_id': 'req-1ac96fe7-65d1-4137-9497-7fbca30f7a09'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:16.634418 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-1ac96fe7-65d1-4137-9497-7fbca30f7a09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5afc5e3b8e1472bb2a6fe8da9d34e11', 'request_id': 'req-1ac96fe7-65d1-4137-9497-7fbca30f7a09'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:16.643157 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-1ac96fe7-65d1-4137-9497-7fbca30f7a09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5afc5e3b8e1472bb2a6fe8da9d34e11', 'request_id': 'req-1ac96fe7-65d1-4137-9497-7fbca30f7a09'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:16.664603 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-1ac96fe7-65d1-4137-9497-7fbca30f7a09 admin admin] The token cache is being invalidate because project e5afc5e3b8e1472bb2a6fe8da9d34e11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:37:16.665221 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-1ac96fe7-65d1-4137-9497-7fbca30f7a09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:16.667761 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-1ac96fe7-65d1-4137-9497-7fbca30f7a09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:16.673325 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 788/1589] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:37:16 2022] DELETE /identity/v3/projects/e5afc5e3b8e1472bb2a6fe8da9d34e11 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:16.674970 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac96fe7-65d1-4137-9497-7fbca30f7a09 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:16.679524 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1ac96fe7-65d1-4137-9497-7fbca30f7a09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:16.714828 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc01515-042a-46bd-846a-4d2da7cc7b0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:16.715454 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc01515-042a-46bd-846a-4d2da7cc7b0e service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:16.717372 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc01515-042a-46bd-846a-4d2da7cc7b0e service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:16.717485 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc01515-042a-46bd-846a-4d2da7cc7b0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:16.723179 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8dc01515-042a-46bd-846a-4d2da7cc7b0e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:16.752112 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:16.800887 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac96fe7-65d1-4137-9497-7fbca30f7a09 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:16.806150 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a300c5-c1ad-4d73-9b45-835cc86a0e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:16.806913 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a300c5-c1ad-4d73-9b45-835cc86a0e06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:16.807459 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a300c5-c1ad-4d73-9b45-835cc86a0e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:16.807848 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a300c5-c1ad-4d73-9b45-835cc86a0e06 admin admin] PATH_INFO: `/v3/projects/3aa3b2710c6548fbabaf2a2095fe0431/users/0bf5747312844825bfaabc33e6228593/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:16.808254 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc01515-042a-46bd-846a-4d2da7cc7b0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:16.809777 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc01515-042a-46bd-846a-4d2da7cc7b0e service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:16.822197 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 802/1590] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:37:16.827279 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc01515-042a-46bd-846a-4d2da7cc7b0e service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:16.829077 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8dc01515-042a-46bd-846a-4d2da7cc7b0e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:16.830301 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a300c5-c1ad-4d73-9b45-835cc86a0e06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3aa3b2710c6548fbabaf2a2095fe0431, user_id=0bf5747312844825bfaabc33e6228593, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:16.834279 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a300c5-c1ad-4d73-9b45-835cc86a0e06 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:16.853431 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc01515-042a-46bd-846a-4d2da7cc7b0e service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:16.854336 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 789/1591] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:37:16 2022] PUT /identity/v3/projects/3aa3b2710c6548fbabaf2a2095fe0431/users/0bf5747312844825bfaabc33e6228593/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:16.865860 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff133598-3b1c-4572-a77d-9c30432b6910 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:16.866288 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff133598-3b1c-4572-a77d-9c30432b6910 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:16.866648 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff133598-3b1c-4572-a77d-9c30432b6910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:16.915695 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-ff133598-3b1c-4572-a77d-9c30432b6910 None None] MFA Rules not processed for user `0bf5747312844825bfaabc33e6228593`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:16.917052 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ff133598-3b1c-4572-a77d-9c30432b6910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:16.928726 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:16.957674 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1af6bb8-554a-4940-9653-3d1aff1721b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:16.958501 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af6bb8-554a-4940-9653-3d1aff1721b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:16.958754 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af6bb8-554a-4940-9653-3d1aff1721b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:16.959019 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af6bb8-554a-4940-9653-3d1aff1721b8 admin admin] PATH_INFO: `/v3/users/9d9284b6a11c4c739851015f449e769a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:16.960422 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1af6bb8-554a-4940-9653-3d1aff1721b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d9284b6a11c4c739851015f449e769a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:16.962928 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1af6bb8-554a-4940-9653-3d1aff1721b8 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:17.032416 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b1af6bb8-554a-4940-9653-3d1aff1721b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d9284b6a11c4c739851015f449e769a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:17.041243 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b1af6bb8-554a-4940-9653-3d1aff1721b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d9284b6a11c4c739851015f449e769a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:17.042328 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 790/1592] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:17.053805 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff133598-3b1c-4572-a77d-9c30432b6910 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:17.056017 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ff133598-3b1c-4572-a77d-9c30432b6910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:17.069646 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b1af6bb8-554a-4940-9653-3d1aff1721b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d9284b6a11c4c739851015f449e769a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:17.082455 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b1af6bb8-554a-4940-9653-3d1aff1721b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d9284b6a11c4c739851015f449e769a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:17.083577 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b1af6bb8-554a-4940-9653-3d1aff1721b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d9284b6a11c4c739851015f449e769a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:17.096632 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 803/1593] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:37:16 2022] DELETE /identity/v3/users/9d9284b6a11c4c739851015f449e769a => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:17.163302 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff133598-3b1c-4572-a77d-9c30432b6910 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:17.243969 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64268ae5-5b04-4821-ada8-8bea9e2a928a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:17.244502 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64268ae5-5b04-4821-ada8-8bea9e2a928a service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:17.244804 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64268ae5-5b04-4821-ada8-8bea9e2a928a service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:17.245117 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64268ae5-5b04-4821-ada8-8bea9e2a928a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:17.247619 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-64268ae5-5b04-4821-ada8-8bea9e2a928a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:17.259440 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:17.359179 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64268ae5-5b04-4821-ada8-8bea9e2a928a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:17.360835 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64268ae5-5b04-4821-ada8-8bea9e2a928a service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:17.370295 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 791/1594] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:17 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:37:17.443999 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1af6bb8-554a-4940-9653-3d1aff1721b8 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:17.445761 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b1af6bb8-554a-4940-9653-3d1aff1721b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:17.539968 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1af6bb8-554a-4940-9653-3d1aff1721b8 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:17.545044 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41030986-2d41-45a1-89d8-d31ce2963e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:17.545688 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41030986-2d41-45a1-89d8-d31ce2963e42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:17.546144 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41030986-2d41-45a1-89d8-d31ce2963e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:17.546397 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41030986-2d41-45a1-89d8-d31ce2963e42 admin admin] PATH_INFO: `/v3/projects/93341c892c6b4d8cbe9145446881b8f4` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:17.547519 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41030986-2d41-45a1-89d8-d31ce2963e42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93341c892c6b4d8cbe9145446881b8f4)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:17.549453 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41030986-2d41-45a1-89d8-d31ce2963e42 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:17.605650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-41030986-2d41-45a1-89d8-d31ce2963e42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93341c892c6b4d8cbe9145446881b8f4', 'request_id': 'req-41030986-2d41-45a1-89d8-d31ce2963e42'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:17.612233 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-41030986-2d41-45a1-89d8-d31ce2963e42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93341c892c6b4d8cbe9145446881b8f4', 'request_id': 'req-41030986-2d41-45a1-89d8-d31ce2963e42'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:17.638663 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-41030986-2d41-45a1-89d8-d31ce2963e42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93341c892c6b4d8cbe9145446881b8f4', 'request_id': 'req-41030986-2d41-45a1-89d8-d31ce2963e42'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:17.648121 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:17.653592 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-41030986-2d41-45a1-89d8-d31ce2963e42 admin admin] The token cache is being invalidate because project 93341c892c6b4d8cbe9145446881b8f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:37:17.653807 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-41030986-2d41-45a1-89d8-d31ce2963e42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:17.655087 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-41030986-2d41-45a1-89d8-d31ce2963e42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:17.657504 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 804/1595] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:37:17 2022] DELETE /identity/v3/projects/93341c892c6b4d8cbe9145446881b8f4 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:17.663506 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636cac90-b6d4-4588-96ba-d91febf3b7c0 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:17.664053 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636cac90-b6d4-4588-96ba-d91febf3b7c0 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:17.664053 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636cac90-b6d4-4588-96ba-d91febf3b7c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:17.694086 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-636cac90-b6d4-4588-96ba-d91febf3b7c0 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:17.695406 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-636cac90-b6d4-4588-96ba-d91febf3b7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:17.780871 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 792/1596] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:17.790627 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63367813-06b5-49c0-9e35-af3b9e7e4f5a None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:17.791145 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63367813-06b5-49c0-9e35-af3b9e7e4f5a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:17.791550 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63367813-06b5-49c0-9e35-af3b9e7e4f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:17.816454 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-63367813-06b5-49c0-9e35-af3b9e7e4f5a None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:17.817720 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-63367813-06b5-49c0-9e35-af3b9e7e4f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:17.829191 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 805/1597] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:17.834315 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636cac90-b6d4-4588-96ba-d91febf3b7c0 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:17.847187 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636cac90-b6d4-4588-96ba-d91febf3b7c0 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:17.851834 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53192977-ec4c-45be-b22b-12b1d41e59e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:17.852452 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53192977-ec4c-45be-b22b-12b1d41e59e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:17.852819 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53192977-ec4c-45be-b22b-12b1d41e59e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:17.853131 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53192977-ec4c-45be-b22b-12b1d41e59e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:17.853740 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53192977-ec4c-45be-b22b-12b1d41e59e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:37:17.854042 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53192977-ec4c-45be-b22b-12b1d41e59e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:17.855650 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53192977-ec4c-45be-b22b-12b1d41e59e0 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:17.872109 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 793/1598] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:37:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:37:17.873050 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63367813-06b5-49c0-9e35-af3b9e7e4f5a None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:17.883142 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63367813-06b5-49c0-9e35-af3b9e7e4f5a None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:17.887828 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5685c7-a785-49d4-a4d3-f37001972643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:17.888495 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5685c7-a785-49d4-a4d3-f37001972643 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:17.888844 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5685c7-a785-49d4-a4d3-f37001972643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:17.889223 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5685c7-a785-49d4-a4d3-f37001972643 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:17.889846 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5685c7-a785-49d4-a4d3-f37001972643 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:17.891709 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5685c7-a785-49d4-a4d3-f37001972643 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:17.925668 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 806/1599] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:37:17 2022] POST /identity/v3/projects => generated 381 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:37:17.929905 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53192977-ec4c-45be-b22b-12b1d41e59e0 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:17.941502 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53192977-ec4c-45be-b22b-12b1d41e59e0 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:18.026774 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcf9b1e-eecd-46e2-b7e8-ceb8675f1a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:18.027403 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf9b1e-eecd-46e2-b7e8-ceb8675f1a93 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:18.027735 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf9b1e-eecd-46e2-b7e8-ceb8675f1a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:18.028013 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf9b1e-eecd-46e2-b7e8-ceb8675f1a93 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:18.028821 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcf9b1e-eecd-46e2-b7e8-ceb8675f1a93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:18.031231 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcf9b1e-eecd-46e2-b7e8-ceb8675f1a93 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:18.062221 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 794/1600] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:37:17 2022] POST /identity/v3/users => generated 291 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:37:18.063047 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5685c7-a785-49d4-a4d3-f37001972643 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:18.074812 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5685c7-a785-49d4-a4d3-f37001972643 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:18.082021 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992a3fde-c522-4118-bbec-a3b5b008b902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:18.082758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a3fde-c522-4118-bbec-a3b5b008b902 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:18.083033 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a3fde-c522-4118-bbec-a3b5b008b902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:18.083441 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a3fde-c522-4118-bbec-a3b5b008b902 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:18.084106 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992a3fde-c522-4118-bbec-a3b5b008b902 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:18.086446 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992a3fde-c522-4118-bbec-a3b5b008b902 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:18.101331 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 807/1601] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 09:37:18.102509 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcf9b1e-eecd-46e2-b7e8-ceb8675f1a93 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:18.113131 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcf9b1e-eecd-46e2-b7e8-ceb8675f1a93 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:18.117885 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1898ca30-7744-454f-9cd5-dcf763fb191a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:18.118616 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898ca30-7744-454f-9cd5-dcf763fb191a admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:18.118868 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898ca30-7744-454f-9cd5-dcf763fb191a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:18.119041 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898ca30-7744-454f-9cd5-dcf763fb191a admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:18.119662 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1898ca30-7744-454f-9cd5-dcf763fb191a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:18.121291 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1898ca30-7744-454f-9cd5-dcf763fb191a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:18.138790 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 795/1602] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:18.143092 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992a3fde-c522-4118-bbec-a3b5b008b902 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:18.163042 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992a3fde-c522-4118-bbec-a3b5b008b902 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:18.169328 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23738aae-30a6-4719-9eab-048def99552d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:18.170070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23738aae-30a6-4719-9eab-048def99552d admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:18.170964 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23738aae-30a6-4719-9eab-048def99552d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:18.171237 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23738aae-30a6-4719-9eab-048def99552d admin admin] PATH_INFO: `/v3/projects/196b9f6cc8b445a4b3a4cc497da8179f/users/c1df6f561db64fe4ae4db67fd7cbd01a/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:18.187473 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23738aae-30a6-4719-9eab-048def99552d admin admin] RBAC: Authorizing `identity:create_grant(project_id=196b9f6cc8b445a4b3a4cc497da8179f, user_id=c1df6f561db64fe4ae4db67fd7cbd01a, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:18.190164 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23738aae-30a6-4719-9eab-048def99552d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:18.205959 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 808/1603] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:37:18 2022] PUT /identity/v3/projects/196b9f6cc8b445a4b3a4cc497da8179f/users/c1df6f561db64fe4ae4db67fd7cbd01a/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:18.218812 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1898ca30-7744-454f-9cd5-dcf763fb191a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:18.220584 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1898ca30-7744-454f-9cd5-dcf763fb191a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:18.222068 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633ca770-3a89-4dc4-8fb3-e144ed6ed4ae None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:18.222458 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633ca770-3a89-4dc4-8fb3-e144ed6ed4ae None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:18.222855 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633ca770-3a89-4dc4-8fb3-e144ed6ed4ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:18.236598 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1898ca30-7744-454f-9cd5-dcf763fb191a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:18.255104 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:18.261363 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-633ca770-3a89-4dc4-8fb3-e144ed6ed4ae None None] MFA Rules not processed for user `c1df6f561db64fe4ae4db67fd7cbd01a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:18.263180 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-633ca770-3a89-4dc4-8fb3-e144ed6ed4ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:18.352821 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 809/1604] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:18.354901 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633ca770-3a89-4dc4-8fb3-e144ed6ed4ae None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:18.371256 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05c3673-bb44-4bc7-937e-b26f384320bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:18.374565 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05c3673-bb44-4bc7-937e-b26f384320bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:18.374856 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05c3673-bb44-4bc7-937e-b26f384320bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:18.375132 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05c3673-bb44-4bc7-937e-b26f384320bb admin admin] PATH_INFO: `/v3/users/fa529017d91d45f08d641224247aa318` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:18.378293 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05c3673-bb44-4bc7-937e-b26f384320bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa529017d91d45f08d641224247aa318)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:18.379898 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05c3673-bb44-4bc7-937e-b26f384320bb admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:18.474646 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a05c3673-bb44-4bc7-937e-b26f384320bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa529017d91d45f08d641224247aa318'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:18.477210 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633ca770-3a89-4dc4-8fb3-e144ed6ed4ae None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:18.483844 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a05c3673-bb44-4bc7-937e-b26f384320bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa529017d91d45f08d641224247aa318'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:18.490816 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2224bb-0945-4297-a621-bf739d27c1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:18.491636 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2224bb-0945-4297-a621-bf739d27c1ef admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:18.492037 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2224bb-0945-4297-a621-bf739d27c1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:18.492561 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2224bb-0945-4297-a621-bf739d27c1ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:18.493284 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2224bb-0945-4297-a621-bf739d27c1ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:18.496971 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2224bb-0945-4297-a621-bf739d27c1ef admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:18.511160 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a05c3673-bb44-4bc7-937e-b26f384320bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa529017d91d45f08d641224247aa318'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:18.522257 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a05c3673-bb44-4bc7-937e-b26f384320bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa529017d91d45f08d641224247aa318'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:18.523053 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a05c3673-bb44-4bc7-937e-b26f384320bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa529017d91d45f08d641224247aa318'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:18.536136 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 796/1605] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:37:18 2022] DELETE /identity/v3/users/fa529017d91d45f08d641224247aa318 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:18.543099 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 810/1606] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:37:18 2022] POST /identity/v3/projects => generated 383 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:37:18.543642 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac76b89-e53e-48b9-ad83-b7c8fb06d73d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:18.543642 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac76b89-e53e-48b9-ad83-b7c8fb06d73d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:18.543642 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac76b89-e53e-48b9-ad83-b7c8fb06d73d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:18.546474 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2224bb-0945-4297-a621-bf739d27c1ef admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:18.549531 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ac2224bb-0945-4297-a621-bf739d27c1ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:18.562569 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-5ac76b89-e53e-48b9-ad83-b7c8fb06d73d None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:18.564076 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5ac76b89-e53e-48b9-ad83-b7c8fb06d73d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:18.569965 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:18.653568 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 797/1607] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:18.655586 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2224bb-0945-4297-a621-bf739d27c1ef admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:18.666375 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac76b89-e53e-48b9-ad83-b7c8fb06d73d None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:18.668091 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5ac76b89-e53e-48b9-ad83-b7c8fb06d73d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:18.674585 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df4bb86-6cad-41bd-99a0-d71c1d0cb17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:18.679973 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df4bb86-6cad-41bd-99a0-d71c1d0cb17e admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:18.680296 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df4bb86-6cad-41bd-99a0-d71c1d0cb17e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:18.680597 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df4bb86-6cad-41bd-99a0-d71c1d0cb17e admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:18.681263 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df4bb86-6cad-41bd-99a0-d71c1d0cb17e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:18.684920 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df4bb86-6cad-41bd-99a0-d71c1d0cb17e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:18.716348 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 811/1608] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:37:18 2022] POST /identity/v3/users => generated 298 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:37:18.723070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df4bb86-6cad-41bd-99a0-d71c1d0cb17e admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:18.736939 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df4bb86-6cad-41bd-99a0-d71c1d0cb17e admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:18.744657 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d38cff-dc06-4446-8af4-81354e74de70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:18.746159 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d38cff-dc06-4446-8af4-81354e74de70 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:18.747021 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d38cff-dc06-4446-8af4-81354e74de70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:18.748748 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d38cff-dc06-4446-8af4-81354e74de70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:18.750852 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d38cff-dc06-4446-8af4-81354e74de70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:18.757096 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d38cff-dc06-4446-8af4-81354e74de70 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:18.761548 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:18.789629 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac76b89-e53e-48b9-ad83-b7c8fb06d73d None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:18.794559 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd29c3b-e354-49bc-99a0-fe2263743963 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:18.795526 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd29c3b-e354-49bc-99a0-fe2263743963 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:18.796031 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd29c3b-e354-49bc-99a0-fe2263743963 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:18.796401 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd29c3b-e354-49bc-99a0-fe2263743963 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:18.799971 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 812/1609] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 09:37:18.804370 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d38cff-dc06-4446-8af4-81354e74de70 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:18.808821 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd29c3b-e354-49bc-99a0-fe2263743963 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:18.810674 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd29c3b-e354-49bc-99a0-fe2263743963 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:18.828725 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d38cff-dc06-4446-8af4-81354e74de70 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:18.836560 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a5721-6f6e-44c9-85a6-e50dcdc8ecd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:18.837369 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a5721-6f6e-44c9-85a6-e50dcdc8ecd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:18.837830 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a5721-6f6e-44c9-85a6-e50dcdc8ecd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:18.838281 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a5721-6f6e-44c9-85a6-e50dcdc8ecd3 admin admin] PATH_INFO: `/v3/projects/e33bba2fc97f45798bd2f9b22d0b4007/users/42d9f6cee397450a932184227ffbee46/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:18.844105 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 798/1610] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:37:18.890812 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9a5721-6f6e-44c9-85a6-e50dcdc8ecd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e33bba2fc97f45798bd2f9b22d0b4007, user_id=42d9f6cee397450a932184227ffbee46, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:18.896235 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9a5721-6f6e-44c9-85a6-e50dcdc8ecd3 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:18.922993 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 813/1611] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:37:18 2022] PUT /identity/v3/projects/e33bba2fc97f45798bd2f9b22d0b4007/users/42d9f6cee397450a932184227ffbee46/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:18.927126 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd29c3b-e354-49bc-99a0-fe2263743963 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:18.958266 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd29c3b-e354-49bc-99a0-fe2263743963 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:19.121167 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a3766a-a16b-412a-a962-dbc9879255d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:19.121167 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a3766a-a16b-412a-a962-dbc9879255d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:19.121167 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a3766a-a16b-412a-a962-dbc9879255d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:19.121167 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a3766a-a16b-412a-a962-dbc9879255d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:19.121527 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a3766a-a16b-412a-a962-dbc9879255d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:19.121527 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a3766a-a16b-412a-a962-dbc9879255d2 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:19.139941 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 799/1612] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:19.141719 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a5721-6f6e-44c9-85a6-e50dcdc8ecd3 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:19.156987 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a5721-6f6e-44c9-85a6-e50dcdc8ecd3 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:19.177374 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd196e5-6a41-45a8-b1c7-1d991f3af430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:19.178380 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd196e5-6a41-45a8-b1c7-1d991f3af430 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:19.178595 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd196e5-6a41-45a8-b1c7-1d991f3af430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:19.178691 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd196e5-6a41-45a8-b1c7-1d991f3af430 admin admin] PATH_INFO: `/v3/projects/e33bba2fc97f45798bd2f9b22d0b4007/users/42d9f6cee397450a932184227ffbee46/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:19.184279 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd196e5-6a41-45a8-b1c7-1d991f3af430 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e33bba2fc97f45798bd2f9b22d0b4007, user_id=42d9f6cee397450a932184227ffbee46, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:19.191749 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd196e5-6a41-45a8-b1c7-1d991f3af430 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:19.231754 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 814/1613] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:37:19 2022] PUT /identity/v3/projects/e33bba2fc97f45798bd2f9b22d0b4007/users/42d9f6cee397450a932184227ffbee46/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:19.242488 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f264452-0be4-4f0b-b462-829acfaff867 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:19.242804 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f264452-0be4-4f0b-b462-829acfaff867 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:19.243041 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f264452-0be4-4f0b-b462-829acfaff867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:19.301764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-9f264452-0be4-4f0b-b462-829acfaff867 None None] MFA Rules not processed for user `42d9f6cee397450a932184227ffbee46`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:19.304762 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9f264452-0be4-4f0b-b462-829acfaff867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:19.407631 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 800/1614] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:19 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:19.419240 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd196e5-6a41-45a8-b1c7-1d991f3af430 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:19.421542 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1cd196e5-6a41-45a8-b1c7-1d991f3af430 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:19.499568 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd196e5-6a41-45a8-b1c7-1d991f3af430 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:19.503952 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327c5025-f87c-4be9-8d5c-9e1e591e4397 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:19.504703 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c5025-f87c-4be9-8d5c-9e1e591e4397 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:19.505099 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c5025-f87c-4be9-8d5c-9e1e591e4397 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:19.505447 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c5025-f87c-4be9-8d5c-9e1e591e4397 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:19.517894 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327c5025-f87c-4be9-8d5c-9e1e591e4397 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:19.519823 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327c5025-f87c-4be9-8d5c-9e1e591e4397 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:19.533571 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 815/1615] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:19 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:37:19.686309 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f264452-0be4-4f0b-b462-829acfaff867 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:19.701615 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f264452-0be4-4f0b-b462-829acfaff867 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:19.711333 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b870dc-26bc-4c25-9627-215af0dfa244 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:19.712465 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b870dc-26bc-4c25-9627-215af0dfa244 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:19.713051 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b870dc-26bc-4c25-9627-215af0dfa244 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:19.715758 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b870dc-26bc-4c25-9627-215af0dfa244 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:19.719248 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-43b870dc-26bc-4c25-9627-215af0dfa244 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:19.851958 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b870dc-26bc-4c25-9627-215af0dfa244 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:19.855131 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b870dc-26bc-4c25-9627-215af0dfa244 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:19.864877 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:19.871375 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 801/1616] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:19 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:37:20.015071 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327c5025-f87c-4be9-8d5c-9e1e591e4397 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:20.017239 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-327c5025-f87c-4be9-8d5c-9e1e591e4397 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:20.141056 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:20.187851 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327c5025-f87c-4be9-8d5c-9e1e591e4397 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:20.196046 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b870dc-26bc-4c25-9627-215af0dfa244 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:20.199051 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-43b870dc-26bc-4c25-9627-215af0dfa244 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:20.199695 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f518f45b-02ed-448e-98d8-49a0ea7f8275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:20.200217 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518f45b-02ed-448e-98d8-49a0ea7f8275 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:20.200468 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518f45b-02ed-448e-98d8-49a0ea7f8275 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:20.200845 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518f45b-02ed-448e-98d8-49a0ea7f8275 admin admin] PATH_INFO: `/v3/projects/857cfc4b6c254af58e41a22fb02b1286` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:20.203472 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f518f45b-02ed-448e-98d8-49a0ea7f8275 admin admin] RBAC: Authorizing `identity:delete_project(project_id=857cfc4b6c254af58e41a22fb02b1286)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:20.206149 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f518f45b-02ed-448e-98d8-49a0ea7f8275 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:20.229489 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b870dc-26bc-4c25-9627-215af0dfa244 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:20.238397 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184fe162-47d0-471b-95aa-faf5f6f1ddb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:20.240283 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184fe162-47d0-471b-95aa-faf5f6f1ddb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:20.241034 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184fe162-47d0-471b-95aa-faf5f6f1ddb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:20.241442 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184fe162-47d0-471b-95aa-faf5f6f1ddb9 admin admin] PATH_INFO: `/v3/users/022d51c1cd8b48d29615ff740209be28` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:20.242886 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184fe162-47d0-471b-95aa-faf5f6f1ddb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=022d51c1cd8b48d29615ff740209be28)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:20.245242 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184fe162-47d0-471b-95aa-faf5f6f1ddb9 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:20.298787 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f518f45b-02ed-448e-98d8-49a0ea7f8275 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '857cfc4b6c254af58e41a22fb02b1286', 'request_id': 'req-f518f45b-02ed-448e-98d8-49a0ea7f8275'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:20.312195 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f518f45b-02ed-448e-98d8-49a0ea7f8275 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '857cfc4b6c254af58e41a22fb02b1286', 'request_id': 'req-f518f45b-02ed-448e-98d8-49a0ea7f8275'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:20.323746 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f518f45b-02ed-448e-98d8-49a0ea7f8275 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '857cfc4b6c254af58e41a22fb02b1286', 'request_id': 'req-f518f45b-02ed-448e-98d8-49a0ea7f8275'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:20.344376 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f518f45b-02ed-448e-98d8-49a0ea7f8275 admin admin] The token cache is being invalidate because project 857cfc4b6c254af58e41a22fb02b1286 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:37:20.345153 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f518f45b-02ed-448e-98d8-49a0ea7f8275 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:20.346202 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f518f45b-02ed-448e-98d8-49a0ea7f8275 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:20.349312 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 816/1617] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:37:20 2022] DELETE /identity/v3/projects/857cfc4b6c254af58e41a22fb02b1286 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:20.352694 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f518f45b-02ed-448e-98d8-49a0ea7f8275 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:20.354761 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f518f45b-02ed-448e-98d8-49a0ea7f8275 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:20.368946 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-184fe162-47d0-471b-95aa-faf5f6f1ddb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '022d51c1cd8b48d29615ff740209be28'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:20.396035 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-184fe162-47d0-471b-95aa-faf5f6f1ddb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '022d51c1cd8b48d29615ff740209be28'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:20.441030 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-184fe162-47d0-471b-95aa-faf5f6f1ddb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '022d51c1cd8b48d29615ff740209be28'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:20.467305 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f518f45b-02ed-448e-98d8-49a0ea7f8275 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:20.474386 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-184fe162-47d0-471b-95aa-faf5f6f1ddb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '022d51c1cd8b48d29615ff740209be28'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:20.475722 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8893503c-ffc9-48eb-bb01-b5ed4614747a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:20.476070 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-184fe162-47d0-471b-95aa-faf5f6f1ddb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '022d51c1cd8b48d29615ff740209be28'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:20.476865 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893503c-ffc9-48eb-bb01-b5ed4614747a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:20.478306 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893503c-ffc9-48eb-bb01-b5ed4614747a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:20.478306 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893503c-ffc9-48eb-bb01-b5ed4614747a admin admin] PATH_INFO: `/v3/users/5994e18e32ae40f394e8971084d913b9` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:20.479616 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8893503c-ffc9-48eb-bb01-b5ed4614747a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5994e18e32ae40f394e8971084d913b9)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:20.482502 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8893503c-ffc9-48eb-bb01-b5ed4614747a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:20.491920 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 802/1618] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:37:20 2022] DELETE /identity/v3/users/022d51c1cd8b48d29615ff740209be28 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:20.499938 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32d657-71f9-47c4-911f-a5318cf0ff83 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:20.505694 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32d657-71f9-47c4-911f-a5318cf0ff83 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:20.507808 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32d657-71f9-47c4-911f-a5318cf0ff83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:20.573999 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-0a32d657-71f9-47c4-911f-a5318cf0ff83 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:20.577368 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0a32d657-71f9-47c4-911f-a5318cf0ff83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:20.647888 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8893503c-ffc9-48eb-bb01-b5ed4614747a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5994e18e32ae40f394e8971084d913b9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:20.668953 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8893503c-ffc9-48eb-bb01-b5ed4614747a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5994e18e32ae40f394e8971084d913b9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:20.703050 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8893503c-ffc9-48eb-bb01-b5ed4614747a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5994e18e32ae40f394e8971084d913b9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:20.715948 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 803/1619] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:20.725922 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8893503c-ffc9-48eb-bb01-b5ed4614747a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5994e18e32ae40f394e8971084d913b9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:20.727763 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8893503c-ffc9-48eb-bb01-b5ed4614747a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5994e18e32ae40f394e8971084d913b9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:20.738454 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 817/1620] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:37:20 2022] DELETE /identity/v3/users/5994e18e32ae40f394e8971084d913b9 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:37:20.774887 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a32d657-71f9-47c4-911f-a5318cf0ff83 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:20.777537 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0a32d657-71f9-47c4-911f-a5318cf0ff83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:20.920928 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a32d657-71f9-47c4-911f-a5318cf0ff83 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:20.967472 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:21.027027 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f70d8-264b-40b4-991b-c2f860a55c59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:21.040618 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f70d8-264b-40b4-991b-c2f860a55c59 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:21.040913 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f70d8-264b-40b4-991b-c2f860a55c59 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:21.041191 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f70d8-264b-40b4-991b-c2f860a55c59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:21.046574 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-769f70d8-264b-40b4-991b-c2f860a55c59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:21.173047 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769f70d8-264b-40b4-991b-c2f860a55c59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:21.175188 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769f70d8-264b-40b4-991b-c2f860a55c59 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:21.188284 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 804/1621] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:37:21.567332 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8893503c-ffc9-48eb-bb01-b5ed4614747a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:21.569991 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8893503c-ffc9-48eb-bb01-b5ed4614747a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:21.599989 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:21.607891 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8893503c-ffc9-48eb-bb01-b5ed4614747a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:21.612790 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f70d8-264b-40b4-991b-c2f860a55c59 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:21.614258 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-769f70d8-264b-40b4-991b-c2f860a55c59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:21.617113 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd60f34-78bd-4f31-9938-9f8b43411ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:21.617481 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd60f34-78bd-4f31-9938-9f8b43411ce3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:21.618111 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd60f34-78bd-4f31-9938-9f8b43411ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:21.618547 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd60f34-78bd-4f31-9938-9f8b43411ce3 admin admin] PATH_INFO: `/v3/projects/5efbf0eeb65d47efa636377c34e2f3d6` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:21.621633 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd60f34-78bd-4f31-9938-9f8b43411ce3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5efbf0eeb65d47efa636377c34e2f3d6)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:21.624203 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd60f34-78bd-4f31-9938-9f8b43411ce3 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:21.627048 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f70d8-264b-40b4-991b-c2f860a55c59 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:21.635244 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb993a-e755-4850-ae17-c3318d603774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:21.636435 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb993a-e755-4850-ae17-c3318d603774 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:21.636916 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb993a-e755-4850-ae17-c3318d603774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:21.637556 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb993a-e755-4850-ae17-c3318d603774 admin admin] PATH_INFO: `/v3/projects/0019184a911946438dbdae1892ac1a06` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:21.640052 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbb993a-e755-4850-ae17-c3318d603774 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0019184a911946438dbdae1892ac1a06)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:21.642687 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbb993a-e755-4850-ae17-c3318d603774 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:21.704980 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5bd60f34-78bd-4f31-9938-9f8b43411ce3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5efbf0eeb65d47efa636377c34e2f3d6', 'request_id': 'req-5bd60f34-78bd-4f31-9938-9f8b43411ce3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:21.716045 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5bd60f34-78bd-4f31-9938-9f8b43411ce3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5efbf0eeb65d47efa636377c34e2f3d6', 'request_id': 'req-5bd60f34-78bd-4f31-9938-9f8b43411ce3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:21.726545 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4fbb993a-e755-4850-ae17-c3318d603774 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0019184a911946438dbdae1892ac1a06', 'request_id': 'req-4fbb993a-e755-4850-ae17-c3318d603774'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:21.727087 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5bd60f34-78bd-4f31-9938-9f8b43411ce3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5efbf0eeb65d47efa636377c34e2f3d6', 'request_id': 'req-5bd60f34-78bd-4f31-9938-9f8b43411ce3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:21.733229 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4fbb993a-e755-4850-ae17-c3318d603774 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0019184a911946438dbdae1892ac1a06', 'request_id': 'req-4fbb993a-e755-4850-ae17-c3318d603774'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:21.739621 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5bd60f34-78bd-4f31-9938-9f8b43411ce3 admin admin] The token cache is being invalidate because project 5efbf0eeb65d47efa636377c34e2f3d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:37:21.739763 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4fbb993a-e755-4850-ae17-c3318d603774 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0019184a911946438dbdae1892ac1a06', 'request_id': 'req-4fbb993a-e755-4850-ae17-c3318d603774'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:21.740234 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5bd60f34-78bd-4f31-9938-9f8b43411ce3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:21.741016 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5bd60f34-78bd-4f31-9938-9f8b43411ce3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:21.742846 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 818/1622] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:37:21 2022] DELETE /identity/v3/projects/5efbf0eeb65d47efa636377c34e2f3d6 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:21.749068 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30b35f-ceb4-4508-81d9-c249b45fdb77 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:21.750575 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30b35f-ceb4-4508-81d9-c249b45fdb77 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:21.750954 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30b35f-ceb4-4508-81d9-c249b45fdb77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:21.761099 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4fbb993a-e755-4850-ae17-c3318d603774 admin admin] The token cache is being invalidate because project 0019184a911946438dbdae1892ac1a06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:37:21.761670 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4fbb993a-e755-4850-ae17-c3318d603774 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:21.762709 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4fbb993a-e755-4850-ae17-c3318d603774 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:21.768318 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 805/1623] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:37:21 2022] DELETE /identity/v3/projects/0019184a911946438dbdae1892ac1a06 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:21.769480 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb993a-e755-4850-ae17-c3318d603774 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:21.771129 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4fbb993a-e755-4850-ae17-c3318d603774 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:21.775719 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-bf30b35f-ceb4-4508-81d9-c249b45fdb77 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:21.777854 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-bf30b35f-ceb4-4508-81d9-c249b45fdb77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:21.855469 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb993a-e755-4850-ae17-c3318d603774 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:21.861342 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbae325b-7a1e-40a0-9891-8fe3d93e8f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:21.862105 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae325b-7a1e-40a0-9891-8fe3d93e8f6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:21.862825 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae325b-7a1e-40a0-9891-8fe3d93e8f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:21.863208 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae325b-7a1e-40a0-9891-8fe3d93e8f6e admin admin] PATH_INFO: `/v3/users/47b37b5864aa4595905ac7fc373b790c` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:21.864976 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbae325b-7a1e-40a0-9891-8fe3d93e8f6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=47b37b5864aa4595905ac7fc373b790c)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:21.867581 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbae325b-7a1e-40a0-9891-8fe3d93e8f6e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:21.870019 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:21.910693 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 819/1624] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:37:21.913630 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f65085-ae24-495e-a7c6-883052a9823c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:21.913910 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f65085-ae24-495e-a7c6-883052a9823c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:21.914220 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f65085-ae24-495e-a7c6-883052a9823c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:21.938098 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-97f65085-ae24-495e-a7c6-883052a9823c None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:21.939430 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-97f65085-ae24-495e-a7c6-883052a9823c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:21.950574 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 820/1625] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:21.966694 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f65085-ae24-495e-a7c6-883052a9823c None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:21.988760 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f65085-ae24-495e-a7c6-883052a9823c None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:21.992590 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-fbae325b-7a1e-40a0-9891-8fe3d93e8f6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47b37b5864aa4595905ac7fc373b790c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:22.004672 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-fbae325b-7a1e-40a0-9891-8fe3d93e8f6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47b37b5864aa4595905ac7fc373b790c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:22.006939 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e581c0d0-c383-40b6-a0e1-4d959f716568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:22.007760 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581c0d0-c383-40b6-a0e1-4d959f716568 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:22.008286 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581c0d0-c383-40b6-a0e1-4d959f716568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:22.008745 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581c0d0-c383-40b6-a0e1-4d959f716568 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:22.009535 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e581c0d0-c383-40b6-a0e1-4d959f716568 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:37:22.010002 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e581c0d0-c383-40b6-a0e1-4d959f716568 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:22.011738 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e581c0d0-c383-40b6-a0e1-4d959f716568 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:22.041091 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-fbae325b-7a1e-40a0-9891-8fe3d93e8f6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47b37b5864aa4595905ac7fc373b790c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:22.045854 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 821/1626] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:37:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:37:22.050356 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e581c0d0-c383-40b6-a0e1-4d959f716568 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:22.052031 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-fbae325b-7a1e-40a0-9891-8fe3d93e8f6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47b37b5864aa4595905ac7fc373b790c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:22.053140 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-fbae325b-7a1e-40a0-9891-8fe3d93e8f6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47b37b5864aa4595905ac7fc373b790c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:22.063127 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 806/1627] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:37:21 2022] DELETE /identity/v3/users/47b37b5864aa4595905ac7fc373b790c => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:22.066153 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e581c0d0-c383-40b6-a0e1-4d959f716568 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:22.138226 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54052106-c99a-4ef1-a123-8715d3b4e29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:22.139563 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54052106-c99a-4ef1-a123-8715d3b4e29a admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:22.139936 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54052106-c99a-4ef1-a123-8715d3b4e29a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:22.140286 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54052106-c99a-4ef1-a123-8715d3b4e29a admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:22.141140 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54052106-c99a-4ef1-a123-8715d3b4e29a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:22.143110 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54052106-c99a-4ef1-a123-8715d3b4e29a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:22.155994 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:22.175367 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 822/1628] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:37:22 2022] POST /identity/v3/projects => generated 385 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:37:22.179873 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbae325b-7a1e-40a0-9891-8fe3d93e8f6e admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:22.182278 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fbae325b-7a1e-40a0-9891-8fe3d93e8f6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:22.196757 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbae325b-7a1e-40a0-9891-8fe3d93e8f6e admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:22.287986 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9636e449-2479-4aed-b549-d27dc96cb3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:22.287986 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636e449-2479-4aed-b549-d27dc96cb3e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:22.287986 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636e449-2479-4aed-b549-d27dc96cb3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:22.287986 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636e449-2479-4aed-b549-d27dc96cb3e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:22.288700 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9636e449-2479-4aed-b549-d27dc96cb3e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:22.291004 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9636e449-2479-4aed-b549-d27dc96cb3e1 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:22.350503 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54052106-c99a-4ef1-a123-8715d3b4e29a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:22.351139 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 807/1629] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:37:22 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:37:22.369673 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54052106-c99a-4ef1-a123-8715d3b4e29a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:22.376025 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b889e0d-c327-4061-b6af-bbba9b0118fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:22.376768 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b889e0d-c327-4061-b6af-bbba9b0118fc admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:22.377120 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b889e0d-c327-4061-b6af-bbba9b0118fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:22.377460 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b889e0d-c327-4061-b6af-bbba9b0118fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:22.379189 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b889e0d-c327-4061-b6af-bbba9b0118fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:22.381410 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b889e0d-c327-4061-b6af-bbba9b0118fc admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:22.396287 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 823/1630] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:22 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:22.401987 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9636e449-2479-4aed-b549-d27dc96cb3e1 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:22.428373 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9636e449-2479-4aed-b549-d27dc96cb3e1 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:22.447085 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da001c69-dcf3-4f54-873d-9821a73fd308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:22.447678 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da001c69-dcf3-4f54-873d-9821a73fd308 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:22.447955 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da001c69-dcf3-4f54-873d-9821a73fd308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:22.448222 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da001c69-dcf3-4f54-873d-9821a73fd308 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:22.448908 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da001c69-dcf3-4f54-873d-9821a73fd308 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:22.450512 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da001c69-dcf3-4f54-873d-9821a73fd308 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:22.474594 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 808/1631] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:22 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:22.478951 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b889e0d-c327-4061-b6af-bbba9b0118fc admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:22.493461 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b889e0d-c327-4061-b6af-bbba9b0118fc admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:22.499564 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe5a67-0c00-4541-be1e-3cdcc5149e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:22.500388 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe5a67-0c00-4541-be1e-3cdcc5149e38 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:22.500686 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe5a67-0c00-4541-be1e-3cdcc5149e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:22.501454 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe5a67-0c00-4541-be1e-3cdcc5149e38 admin admin] PATH_INFO: `/v3/projects/1cf82af3dca14828a82f164f538668a5/users/5f4da0edd6544ad9846e32dc0502e962/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:22.521119 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fe5a67-0c00-4541-be1e-3cdcc5149e38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cf82af3dca14828a82f164f538668a5, user_id=5f4da0edd6544ad9846e32dc0502e962, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:22.524585 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fe5a67-0c00-4541-be1e-3cdcc5149e38 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:22.543665 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 824/1632] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:37:22 2022] PUT /identity/v3/projects/1cf82af3dca14828a82f164f538668a5/users/5f4da0edd6544ad9846e32dc0502e962/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:22.556746 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7391926e-ecf5-4bb1-9206-d33096df3b0c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:22.557173 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7391926e-ecf5-4bb1-9206-d33096df3b0c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:22.557681 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7391926e-ecf5-4bb1-9206-d33096df3b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:22.590629 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe5a67-0c00-4541-be1e-3cdcc5149e38 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:22.593175 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a5fe5a67-0c00-4541-be1e-3cdcc5149e38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:22.613240 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-7391926e-ecf5-4bb1-9206-d33096df3b0c None None] MFA Rules not processed for user `5f4da0edd6544ad9846e32dc0502e962`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:22.614474 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7391926e-ecf5-4bb1-9206-d33096df3b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:22.681898 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:22.692073 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe5a67-0c00-4541-be1e-3cdcc5149e38 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:22.703956 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c2490a-a8a5-4597-8e5e-a2d2fe1152c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:22.704775 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c2490a-a8a5-4597-8e5e-a2d2fe1152c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:22.705046 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c2490a-a8a5-4597-8e5e-a2d2fe1152c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:22.705717 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c2490a-a8a5-4597-8e5e-a2d2fe1152c0 admin admin] PATH_INFO: `/v3/projects/317401fd1d67455d9a80b629a92de188` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:22.707131 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c2490a-a8a5-4597-8e5e-a2d2fe1152c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=317401fd1d67455d9a80b629a92de188)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:22.709546 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c2490a-a8a5-4597-8e5e-a2d2fe1152c0 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:22.722624 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 809/1633] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:22.723978 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7391926e-ecf5-4bb1-9206-d33096df3b0c None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:22.738484 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7391926e-ecf5-4bb1-9206-d33096df3b0c None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:22.744321 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f54a84-7f78-4113-a6f0-29db55b60524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:22.745559 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f54a84-7f78-4113-a6f0-29db55b60524 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:22.746556 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f54a84-7f78-4113-a6f0-29db55b60524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:22.746893 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f54a84-7f78-4113-a6f0-29db55b60524 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:22.747600 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f54a84-7f78-4113-a6f0-29db55b60524 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:22.751684 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f54a84-7f78-4113-a6f0-29db55b60524 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:22.765235 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-17c2490a-a8a5-4597-8e5e-a2d2fe1152c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '317401fd1d67455d9a80b629a92de188', 'request_id': 'req-17c2490a-a8a5-4597-8e5e-a2d2fe1152c0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:22.771429 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-17c2490a-a8a5-4597-8e5e-a2d2fe1152c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '317401fd1d67455d9a80b629a92de188', 'request_id': 'req-17c2490a-a8a5-4597-8e5e-a2d2fe1152c0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:22.779830 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-17c2490a-a8a5-4597-8e5e-a2d2fe1152c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '317401fd1d67455d9a80b629a92de188', 'request_id': 'req-17c2490a-a8a5-4597-8e5e-a2d2fe1152c0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:22.788750 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 810/1634] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:37:22 2022] POST /identity/v3/projects => generated 385 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:37:22.791589 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f54a84-7f78-4113-a6f0-29db55b60524 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:22.793311 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-17c2490a-a8a5-4597-8e5e-a2d2fe1152c0 admin admin] The token cache is being invalidate because project 317401fd1d67455d9a80b629a92de188 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:37:22.793583 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-17c2490a-a8a5-4597-8e5e-a2d2fe1152c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:22.794577 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-17c2490a-a8a5-4597-8e5e-a2d2fe1152c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:22.796274 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 825/1635] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:37:22 2022] DELETE /identity/v3/projects/317401fd1d67455d9a80b629a92de188 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:22.801102 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c2490a-a8a5-4597-8e5e-a2d2fe1152c0 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:22.802524 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-17c2490a-a8a5-4597-8e5e-a2d2fe1152c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:22.860467 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:22.870613 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f54a84-7f78-4113-a6f0-29db55b60524 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:22.876288 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e265f-43eb-4f16-9943-8a828ee6ad83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:22.876508 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e265f-43eb-4f16-9943-8a828ee6ad83 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:22.876758 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e265f-43eb-4f16-9943-8a828ee6ad83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:22.877026 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e265f-43eb-4f16-9943-8a828ee6ad83 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:22.877658 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634e265f-43eb-4f16-9943-8a828ee6ad83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:22.879568 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634e265f-43eb-4f16-9943-8a828ee6ad83 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:22.880025 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c2490a-a8a5-4597-8e5e-a2d2fe1152c0 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:22.885571 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0af981-f28d-40d2-b365-21e9c9529051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:22.886341 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0af981-f28d-40d2-b365-21e9c9529051 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:22.886690 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0af981-f28d-40d2-b365-21e9c9529051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:22.887039 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0af981-f28d-40d2-b365-21e9c9529051 admin admin] PATH_INFO: `/v3/users/6f8ae5f9e6274665be8f6cf17c740115` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:22.888251 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0af981-f28d-40d2-b365-21e9c9529051 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f8ae5f9e6274665be8f6cf17c740115)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:22.891579 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0af981-f28d-40d2-b365-21e9c9529051 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:22.906322 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 811/1636] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:37:22 2022] POST /identity/v3/users => generated 299 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:37:22.907466 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6fb454-6fc6-45e4-9bf2-86d041910459 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:22.907748 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6fb454-6fc6-45e4-9bf2-86d041910459 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:22.908010 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6fb454-6fc6-45e4-9bf2-86d041910459 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:22.909336 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 812/1637] 10.222.0.70 () {54 vars in 921 bytes} [Thu Oct 27 09:37:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:37:22.909900 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6fb454-6fc6-45e4-9bf2-86d041910459 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:22.911919 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ac6fb454-6fc6-45e4-9bf2-86d041910459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:22.929490 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6fb454-6fc6-45e4-9bf2-86d041910459 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:22.934469 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635dd1d8-357f-4441-8b8c-42a1af1ca95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:22.935073 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635dd1d8-357f-4441-8b8c-42a1af1ca95b admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:22.935912 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635dd1d8-357f-4441-8b8c-42a1af1ca95b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:22.936187 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635dd1d8-357f-4441-8b8c-42a1af1ca95b admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:22.936936 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635dd1d8-357f-4441-8b8c-42a1af1ca95b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:22.938527 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635dd1d8-357f-4441-8b8c-42a1af1ca95b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:22.953010 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 813/1638] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:22 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:22.953010 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc0e621-a202-4d48-9894-f9bf2f8690e2 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:22.953010 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc0e621-a202-4d48-9894-f9bf2f8690e2 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:22.953010 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc0e621-a202-4d48-9894-f9bf2f8690e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:22.957559 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5f0af981-f28d-40d2-b365-21e9c9529051 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f8ae5f9e6274665be8f6cf17c740115'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:22.966183 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5f0af981-f28d-40d2-b365-21e9c9529051 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f8ae5f9e6274665be8f6cf17c740115'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:22.971703 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-dfc0e621-a202-4d48-9894-f9bf2f8690e2 None None] MFA Rules not processed for user `dbe4a6e4b6184826b30ac4fc13ad9b1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:22.973016 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-dfc0e621-a202-4d48-9894-f9bf2f8690e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:22.980386 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5f0af981-f28d-40d2-b365-21e9c9529051 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f8ae5f9e6274665be8f6cf17c740115'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:22.994268 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5f0af981-f28d-40d2-b365-21e9c9529051 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f8ae5f9e6274665be8f6cf17c740115'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:22.995534 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5f0af981-f28d-40d2-b365-21e9c9529051 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f8ae5f9e6274665be8f6cf17c740115'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:23.004432 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 826/1639] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:37:22 2022] DELETE /identity/v3/users/6f8ae5f9e6274665be8f6cf17c740115 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:23.005659 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0af981-f28d-40d2-b365-21e9c9529051 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:23.015086 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5f0af981-f28d-40d2-b365-21e9c9529051 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:23.074193 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 814/1640] 10.222.0.70 () {58 vars in 1018 bytes} [Thu Oct 27 09:37:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:23.074958 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc0e621-a202-4d48-9894-f9bf2f8690e2 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:23.080961 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-dfc0e621-a202-4d48-9894-f9bf2f8690e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:23.097666 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc0e621-a202-4d48-9894-f9bf2f8690e2 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:23.120089 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0af981-f28d-40d2-b365-21e9c9529051 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:23.127327 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23aba2b-c0f2-45ce-b3e1-005c11388ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:23.128964 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23aba2b-c0f2-45ce-b3e1-005c11388ade admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:23.129460 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23aba2b-c0f2-45ce-b3e1-005c11388ade admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:23.137490 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:23.141274 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23aba2b-c0f2-45ce-b3e1-005c11388ade admin admin] PATH_INFO: `/v3/projects/4c5c9ffc26c74c0d92495ef7653318ce/users/e5083e3ac6854a41bf1fc5b98fa8613d/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:23.176337 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf09dc3-cd38-4ada-aeeb-b7467befa4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:23.178311 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf09dc3-cd38-4ada-aeeb-b7467befa4a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:23.178590 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf09dc3-cd38-4ada-aeeb-b7467befa4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:23.179262 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23aba2b-c0f2-45ce-b3e1-005c11388ade admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c5c9ffc26c74c0d92495ef7653318ce, user_id=e5083e3ac6854a41bf1fc5b98fa8613d, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:23.179969 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf09dc3-cd38-4ada-aeeb-b7467befa4a2 admin admin] PATH_INFO: `/v3/users/c1df6f561db64fe4ae4db67fd7cbd01a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:23.182223 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf09dc3-cd38-4ada-aeeb-b7467befa4a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1df6f561db64fe4ae4db67fd7cbd01a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:23.185353 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23aba2b-c0f2-45ce-b3e1-005c11388ade admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:23.187783 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf09dc3-cd38-4ada-aeeb-b7467befa4a2 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:23.229673 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 827/1641] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:37:23 2022] PUT /identity/v3/projects/4c5c9ffc26c74c0d92495ef7653318ce/users/e5083e3ac6854a41bf1fc5b98fa8613d/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:23.231829 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23aba2b-c0f2-45ce-b3e1-005c11388ade admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:23.238699 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b23aba2b-c0f2-45ce-b3e1-005c11388ade admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:23.264898 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-fdf09dc3-cd38-4ada-aeeb-b7467befa4a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1df6f561db64fe4ae4db67fd7cbd01a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:23.278179 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-fdf09dc3-cd38-4ada-aeeb-b7467befa4a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1df6f561db64fe4ae4db67fd7cbd01a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:23.309872 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-fdf09dc3-cd38-4ada-aeeb-b7467befa4a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1df6f561db64fe4ae4db67fd7cbd01a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:23.321651 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:23.336234 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-fdf09dc3-cd38-4ada-aeeb-b7467befa4a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1df6f561db64fe4ae4db67fd7cbd01a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:23.338295 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-fdf09dc3-cd38-4ada-aeeb-b7467befa4a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1df6f561db64fe4ae4db67fd7cbd01a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:23.351562 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 815/1642] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:37:23 2022] DELETE /identity/v3/users/c1df6f561db64fe4ae4db67fd7cbd01a => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:23.353195 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf09dc3-cd38-4ada-aeeb-b7467befa4a2 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:23.354703 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fdf09dc3-cd38-4ada-aeeb-b7467befa4a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:23.371636 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23aba2b-c0f2-45ce-b3e1-005c11388ade admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:23.483635 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf09dc3-cd38-4ada-aeeb-b7467befa4a2 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:23.490655 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c8b56c-b197-45ef-9298-abe67c31bc3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:23.491411 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8b56c-b197-45ef-9298-abe67c31bc3b admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:23.491695 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8b56c-b197-45ef-9298-abe67c31bc3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:23.491964 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8b56c-b197-45ef-9298-abe67c31bc3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:23.492762 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c8b56c-b197-45ef-9298-abe67c31bc3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:23.495491 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c8b56c-b197-45ef-9298-abe67c31bc3b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:23.516370 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 816/1643] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:23 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:23.521026 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621b157-e7a1-478f-bebf-b41baa1fb167 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:23.521382 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621b157-e7a1-478f-bebf-b41baa1fb167 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:23.521746 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621b157-e7a1-478f-bebf-b41baa1fb167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:23.539141 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcc3e43-5041-47e4-94fc-02bd375546f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe4a6e4b6184826b30ac4fc13ad9b1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:23.541329 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc3e43-5041-47e4-94fc-02bd375546f9 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:23.541329 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc3e43-5041-47e4-94fc-02bd375546f9 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:23.541603 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc3e43-5041-47e4-94fc-02bd375546f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:23.546355 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3bcc3e43-5041-47e4-94fc-02bd375546f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:23.554613 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-8621b157-e7a1-478f-bebf-b41baa1fb167 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:23.556160 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8621b157-e7a1-478f-bebf-b41baa1fb167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:23.573023 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcc3e43-5041-47e4-94fc-02bd375546f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:23.574847 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcc3e43-5041-47e4-94fc-02bd375546f9 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:23.578245 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 817/1644] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:23.582715 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8621b157-e7a1-478f-bebf-b41baa1fb167 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:23.584648 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8621b157-e7a1-478f-bebf-b41baa1fb167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:23.595185 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 828/1645] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:37:23.596559 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcc3e43-5041-47e4-94fc-02bd375546f9 service nova] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:23.610170 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcc3e43-5041-47e4-94fc-02bd375546f9 service nova] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:23.616189 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea16c62-97da-4bca-a065-bccad8f2f00a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:23.616974 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea16c62-97da-4bca-a065-bccad8f2f00a admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:23.618370 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea16c62-97da-4bca-a065-bccad8f2f00a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:23.619043 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea16c62-97da-4bca-a065-bccad8f2f00a admin admin] PATH_INFO: `/v3/projects/4c5c9ffc26c74c0d92495ef7653318ce/users/e5083e3ac6854a41bf1fc5b98fa8613d/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:23.620929 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8621b157-e7a1-478f-bebf-b41baa1fb167 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:23.621323 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea16c62-97da-4bca-a065-bccad8f2f00a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c5c9ffc26c74c0d92495ef7653318ce, user_id=e5083e3ac6854a41bf1fc5b98fa8613d, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:23.625892 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea16c62-97da-4bca-a065-bccad8f2f00a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:23.639573 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 829/1646] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:37:23 2022] PUT /identity/v3/projects/4c5c9ffc26c74c0d92495ef7653318ce/users/e5083e3ac6854a41bf1fc5b98fa8613d/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:23.640481 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7aa7de-ec39-49f3-812a-7cfa3e171578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:23.641368 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7aa7de-ec39-49f3-812a-7cfa3e171578 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:23.641802 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7aa7de-ec39-49f3-812a-7cfa3e171578 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:23.642224 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7aa7de-ec39-49f3-812a-7cfa3e171578 admin admin] PATH_INFO: `/v3/projects/9ca3d22f212346daa6c39ea6ef03d553` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:23.645200 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7aa7de-ec39-49f3-812a-7cfa3e171578 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ca3d22f212346daa6c39ea6ef03d553)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:23.645786 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea16c62-97da-4bca-a065-bccad8f2f00a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:23.649692 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7ea16c62-97da-4bca-a065-bccad8f2f00a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:23.654075 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7aa7de-ec39-49f3-812a-7cfa3e171578 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:23.761698 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:23.772006 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5f7aa7de-ec39-49f3-812a-7cfa3e171578 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ca3d22f212346daa6c39ea6ef03d553', 'request_id': 'req-5f7aa7de-ec39-49f3-812a-7cfa3e171578'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:23.781295 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea16c62-97da-4bca-a065-bccad8f2f00a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:23.793646 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5f7aa7de-ec39-49f3-812a-7cfa3e171578 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ca3d22f212346daa6c39ea6ef03d553', 'request_id': 'req-5f7aa7de-ec39-49f3-812a-7cfa3e171578'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:23.822577 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5f7aa7de-ec39-49f3-812a-7cfa3e171578 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ca3d22f212346daa6c39ea6ef03d553', 'request_id': 'req-5f7aa7de-ec39-49f3-812a-7cfa3e171578'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:23.858161 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5f7aa7de-ec39-49f3-812a-7cfa3e171578 admin admin] The token cache is being invalidate because project 9ca3d22f212346daa6c39ea6ef03d553 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:37:23.858454 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5f7aa7de-ec39-49f3-812a-7cfa3e171578 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:23.859378 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5f7aa7de-ec39-49f3-812a-7cfa3e171578 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:23.861881 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 818/1647] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:37:23 2022] DELETE /identity/v3/projects/9ca3d22f212346daa6c39ea6ef03d553 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:23.866206 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b473b4a0-8a54-4cdb-91c1-ea786aabc708 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:23.866206 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b473b4a0-8a54-4cdb-91c1-ea786aabc708 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:23.866206 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b473b4a0-8a54-4cdb-91c1-ea786aabc708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:23.869760 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788050c3-2f2a-4451-af6e-af140097d26f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:23.870946 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788050c3-2f2a-4451-af6e-af140097d26f service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:23.871603 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788050c3-2f2a-4451-af6e-af140097d26f service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:23.872096 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788050c3-2f2a-4451-af6e-af140097d26f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:23.875114 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-788050c3-2f2a-4451-af6e-af140097d26f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:23.901471 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b473b4a0-8a54-4cdb-91c1-ea786aabc708 None None] MFA Rules not processed for user `e5083e3ac6854a41bf1fc5b98fa8613d`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:23.905512 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b473b4a0-8a54-4cdb-91c1-ea786aabc708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:23.945344 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:24.011236 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788050c3-2f2a-4451-af6e-af140097d26f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:24.014950 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788050c3-2f2a-4451-af6e-af140097d26f service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:24.041498 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 830/1648] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:37:24.042971 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b5de9-746d-47be-9976-95ac4b38310e None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:24.043236 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b5de9-746d-47be-9976-95ac4b38310e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:24.043502 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b5de9-746d-47be-9976-95ac4b38310e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:24.066908 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-b60b5de9-746d-47be-9976-95ac4b38310e None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:24.068455 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b60b5de9-746d-47be-9976-95ac4b38310e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:24.074612 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 819/1649] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:23 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:24.080541 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 831/1650] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:24.086771 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b473b4a0-8a54-4cdb-91c1-ea786aabc708 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:24.088316 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b473b4a0-8a54-4cdb-91c1-ea786aabc708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:24.089582 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5f442f-a733-4509-8a85-60a34f688a9f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:24.089886 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5f442f-a733-4509-8a85-60a34f688a9f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:24.090161 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5f442f-a733-4509-8a85-60a34f688a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:24.114127 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-3b5f442f-a733-4509-8a85-60a34f688a9f None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:24.115335 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3b5f442f-a733-4509-8a85-60a34f688a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:24.128641 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 832/1651] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:37:24.129872 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5f442f-a733-4509-8a85-60a34f688a9f None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:24.140468 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5f442f-a733-4509-8a85-60a34f688a9f None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:24.145208 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18ae804-b122-43f9-a9a0-6e32bd435193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:24.145848 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18ae804-b122-43f9-a9a0-6e32bd435193 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:24.146128 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18ae804-b122-43f9-a9a0-6e32bd435193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:24.146400 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18ae804-b122-43f9-a9a0-6e32bd435193 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:24.147043 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18ae804-b122-43f9-a9a0-6e32bd435193 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:37:24.147343 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18ae804-b122-43f9-a9a0-6e32bd435193 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:24.150479 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18ae804-b122-43f9-a9a0-6e32bd435193 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:24.168191 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 833/1652] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:37:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:37:24.169758 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b473b4a0-8a54-4cdb-91c1-ea786aabc708 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:24.174669 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bcb216-2729-4fc4-a9a2-decb7c3d86ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:24.175373 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bcb216-2729-4fc4-a9a2-decb7c3d86ba service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:24.175650 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bcb216-2729-4fc4-a9a2-decb7c3d86ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:24.175929 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bcb216-2729-4fc4-a9a2-decb7c3d86ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:24.178303 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-64bcb216-2729-4fc4-a9a2-decb7c3d86ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:24.179154 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18ae804-b122-43f9-a9a0-6e32bd435193 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:24.196137 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18ae804-b122-43f9-a9a0-6e32bd435193 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:24.202537 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1012520a-62dc-4925-bfd8-ae5ebd7b2ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:24.203285 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1012520a-62dc-4925-bfd8-ae5ebd7b2ca5 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:24.203971 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1012520a-62dc-4925-bfd8-ae5ebd7b2ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:24.203971 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1012520a-62dc-4925-bfd8-ae5ebd7b2ca5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:24.204882 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1012520a-62dc-4925-bfd8-ae5ebd7b2ca5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:24.207323 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1012520a-62dc-4925-bfd8-ae5ebd7b2ca5 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:24.239676 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 834/1653] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:37:24 2022] POST /identity/v3/projects => generated 381 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:37:24.244606 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1012520a-62dc-4925-bfd8-ae5ebd7b2ca5 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:24.256932 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bcb216-2729-4fc4-a9a2-decb7c3d86ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:24.259059 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bcb216-2729-4fc4-a9a2-decb7c3d86ba service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:24.276383 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 820/1654] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:37:24.349901 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1012520a-62dc-4925-bfd8-ae5ebd7b2ca5 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:24.366309 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3cd08-23ca-4743-a657-fc9437ecafd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:24.367393 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3cd08-23ca-4743-a657-fc9437ecafd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:24.368368 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3cd08-23ca-4743-a657-fc9437ecafd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:24.368833 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3cd08-23ca-4743-a657-fc9437ecafd2 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:24.369612 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d3cd08-23ca-4743-a657-fc9437ecafd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:24.372352 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d3cd08-23ca-4743-a657-fc9437ecafd2 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:24.405278 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 835/1655] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:37:24 2022] POST /identity/v3/users => generated 291 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:37:24.409101 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bcb216-2729-4fc4-a9a2-decb7c3d86ba service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:24.421115 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bcb216-2729-4fc4-a9a2-decb7c3d86ba service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:24.427504 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3746bf-f70a-40af-9465-1ef03715e93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:24.428120 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3746bf-f70a-40af-9465-1ef03715e93f admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:24.428509 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3746bf-f70a-40af-9465-1ef03715e93f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:24.428810 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3746bf-f70a-40af-9465-1ef03715e93f admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:24.429446 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3746bf-f70a-40af-9465-1ef03715e93f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:24.431033 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3746bf-f70a-40af-9465-1ef03715e93f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:24.432863 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:24.451355 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 821/1656] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:24 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:24.457181 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3cd08-23ca-4743-a657-fc9437ecafd2 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:24.469076 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3cd08-23ca-4743-a657-fc9437ecafd2 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:24.480563 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1a5a12-22df-4796-9894-1c4df0aee018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:24.481182 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a5a12-22df-4796-9894-1c4df0aee018 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:24.481463 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a5a12-22df-4796-9894-1c4df0aee018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:24.481732 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a5a12-22df-4796-9894-1c4df0aee018 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:24.485470 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1a5a12-22df-4796-9894-1c4df0aee018 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:24.488134 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1a5a12-22df-4796-9894-1c4df0aee018 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:24.507644 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3746bf-f70a-40af-9465-1ef03715e93f admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:24.516726 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 836/1657] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:24 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:24.518568 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3746bf-f70a-40af-9465-1ef03715e93f admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:24.523519 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca442a1-0e07-4f03-8a07-d511573c3d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:24.524288 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca442a1-0e07-4f03-8a07-d511573c3d9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:24.524462 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca442a1-0e07-4f03-8a07-d511573c3d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:24.524742 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca442a1-0e07-4f03-8a07-d511573c3d9d admin admin] PATH_INFO: `/v3/projects/c96d33b2dd654bd1bd2be63dac9682b7/users/fa6ee0920cc540a591cf4a7e5c150f98/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:24.547060 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca442a1-0e07-4f03-8a07-d511573c3d9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c96d33b2dd654bd1bd2be63dac9682b7, user_id=fa6ee0920cc540a591cf4a7e5c150f98, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:24.551562 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca442a1-0e07-4f03-8a07-d511573c3d9d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:24.586015 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 822/1658] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:37:24 2022] PUT /identity/v3/projects/c96d33b2dd654bd1bd2be63dac9682b7/users/fa6ee0920cc540a591cf4a7e5c150f98/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:24.597914 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cacd143-dccb-457b-884b-b427893d038e None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:24.598457 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cacd143-dccb-457b-884b-b427893d038e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:24.598548 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cacd143-dccb-457b-884b-b427893d038e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:24.637878 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca442a1-0e07-4f03-8a07-d511573c3d9d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:24.640590 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7ca442a1-0e07-4f03-8a07-d511573c3d9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:24.644266 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-2cacd143-dccb-457b-884b-b427893d038e None None] MFA Rules not processed for user `fa6ee0920cc540a591cf4a7e5c150f98`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:24.646113 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2cacd143-dccb-457b-884b-b427893d038e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:24.742083 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:24.776281 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 837/1659] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:24 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:24.782324 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cacd143-dccb-457b-884b-b427893d038e None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:24.785010 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca442a1-0e07-4f03-8a07-d511573c3d9d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:24.796168 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cacd143-dccb-457b-884b-b427893d038e None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:24.798512 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce17fc8c-4562-42d5-b109-3c7ea2ca4f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:24.799523 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce17fc8c-4562-42d5-b109-3c7ea2ca4f75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:24.799995 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce17fc8c-4562-42d5-b109-3c7ea2ca4f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:24.801068 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce17fc8c-4562-42d5-b109-3c7ea2ca4f75 admin admin] PATH_INFO: `/v3/projects/196b9f6cc8b445a4b3a4cc497da8179f` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:24.804042 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce17fc8c-4562-42d5-b109-3c7ea2ca4f75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=196b9f6cc8b445a4b3a4cc497da8179f)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:24.805720 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba84652-6396-4f38-b1d5-baae70a67f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:24.806347 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba84652-6396-4f38-b1d5-baae70a67f0e admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:24.806629 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba84652-6396-4f38-b1d5-baae70a67f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:24.806902 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba84652-6396-4f38-b1d5-baae70a67f0e admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:24.809074 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba84652-6396-4f38-b1d5-baae70a67f0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:24.809712 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce17fc8c-4562-42d5-b109-3c7ea2ca4f75 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:24.811012 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba84652-6396-4f38-b1d5-baae70a67f0e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:24.883330 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 838/1660] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:37:24 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:37:24.887904 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba84652-6396-4f38-b1d5-baae70a67f0e admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:24.918557 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ce17fc8c-4562-42d5-b109-3c7ea2ca4f75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '196b9f6cc8b445a4b3a4cc497da8179f', 'request_id': 'req-ce17fc8c-4562-42d5-b109-3c7ea2ca4f75'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:24.931743 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ce17fc8c-4562-42d5-b109-3c7ea2ca4f75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '196b9f6cc8b445a4b3a4cc497da8179f', 'request_id': 'req-ce17fc8c-4562-42d5-b109-3c7ea2ca4f75'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:24.941842 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ce17fc8c-4562-42d5-b109-3c7ea2ca4f75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '196b9f6cc8b445a4b3a4cc497da8179f', 'request_id': 'req-ce17fc8c-4562-42d5-b109-3c7ea2ca4f75'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:24.960025 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ce17fc8c-4562-42d5-b109-3c7ea2ca4f75 admin admin] The token cache is being invalidate because project 196b9f6cc8b445a4b3a4cc497da8179f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:37:24.960379 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ce17fc8c-4562-42d5-b109-3c7ea2ca4f75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:24.961815 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ce17fc8c-4562-42d5-b109-3c7ea2ca4f75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:24.966089 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 823/1661] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:37:24 2022] DELETE /identity/v3/projects/196b9f6cc8b445a4b3a4cc497da8179f => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:24.969152 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce17fc8c-4562-42d5-b109-3c7ea2ca4f75 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:24.971042 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ce17fc8c-4562-42d5-b109-3c7ea2ca4f75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:24.996619 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba84652-6396-4f38-b1d5-baae70a67f0e admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:25.043544 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce17fc8c-4562-42d5-b109-3c7ea2ca4f75 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:25.048465 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0fe38c-9de3-4eb0-985e-e10b4820cec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:25.049191 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0fe38c-9de3-4eb0-985e-e10b4820cec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:25.049540 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0fe38c-9de3-4eb0-985e-e10b4820cec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:25.049877 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0fe38c-9de3-4eb0-985e-e10b4820cec9 admin admin] PATH_INFO: `/v3/users/42d9f6cee397450a932184227ffbee46` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:25.051093 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0fe38c-9de3-4eb0-985e-e10b4820cec9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42d9f6cee397450a932184227ffbee46)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:25.053303 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0fe38c-9de3-4eb0-985e-e10b4820cec9 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:25.069757 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:25.072296 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70e506-b036-4fde-b110-6d3bcc7b3699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:25.073725 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70e506-b036-4fde-b110-6d3bcc7b3699 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:25.074178 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70e506-b036-4fde-b110-6d3bcc7b3699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:25.078917 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70e506-b036-4fde-b110-6d3bcc7b3699 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:25.079741 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e70e506-b036-4fde-b110-6d3bcc7b3699 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:25.082142 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e70e506-b036-4fde-b110-6d3bcc7b3699 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:25.118534 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 839/1662] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:37:24 2022] POST /identity/v3/users => generated 296 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:37:25.121561 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70e506-b036-4fde-b110-6d3bcc7b3699 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:25.124867 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5e70e506-b036-4fde-b110-6d3bcc7b3699 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:25.137973 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70e506-b036-4fde-b110-6d3bcc7b3699 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:25.143952 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297922bc-4c11-46ff-bf8b-f553c926014a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:25.145077 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297922bc-4c11-46ff-bf8b-f553c926014a admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:25.145263 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297922bc-4c11-46ff-bf8b-f553c926014a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:25.145542 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297922bc-4c11-46ff-bf8b-f553c926014a admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:25.146339 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297922bc-4c11-46ff-bf8b-f553c926014a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:25.148034 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297922bc-4c11-46ff-bf8b-f553c926014a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:25.162294 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4a0fe38c-9de3-4eb0-985e-e10b4820cec9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42d9f6cee397450a932184227ffbee46'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:25.164664 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 840/1663] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:25 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 09:37:25.167815 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297922bc-4c11-46ff-bf8b-f553c926014a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:25.180711 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4a0fe38c-9de3-4eb0-985e-e10b4820cec9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42d9f6cee397450a932184227ffbee46'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:25.186604 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297922bc-4c11-46ff-bf8b-f553c926014a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:25.192320 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c3c7f1-0a8d-47b4-9091-1ab50914413b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:25.192852 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3c7f1-0a8d-47b4-9091-1ab50914413b admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:25.193229 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3c7f1-0a8d-47b4-9091-1ab50914413b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:25.193377 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3c7f1-0a8d-47b4-9091-1ab50914413b admin admin] PATH_INFO: `/v3/projects/d68cd6eb18ae469fa0e3167225d148e8/users/89fea20ab4bf4c2dbebb68496beb6c4b/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:25.204086 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4a0fe38c-9de3-4eb0-985e-e10b4820cec9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42d9f6cee397450a932184227ffbee46'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:25.214814 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c3c7f1-0a8d-47b4-9091-1ab50914413b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d68cd6eb18ae469fa0e3167225d148e8, user_id=89fea20ab4bf4c2dbebb68496beb6c4b, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:25.218088 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c3c7f1-0a8d-47b4-9091-1ab50914413b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:25.221352 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4a0fe38c-9de3-4eb0-985e-e10b4820cec9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42d9f6cee397450a932184227ffbee46'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:25.222134 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4a0fe38c-9de3-4eb0-985e-e10b4820cec9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42d9f6cee397450a932184227ffbee46'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:25.230074 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 824/1664] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:37:24 2022] DELETE /identity/v3/users/42d9f6cee397450a932184227ffbee46 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:37:25.234922 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 841/1665] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:37:25 2022] PUT /identity/v3/projects/d68cd6eb18ae469fa0e3167225d148e8/users/89fea20ab4bf4c2dbebb68496beb6c4b/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:25.238774 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0fe38c-9de3-4eb0-985e-e10b4820cec9 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:25.240360 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4a0fe38c-9de3-4eb0-985e-e10b4820cec9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:25.337329 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0fe38c-9de3-4eb0-985e-e10b4820cec9 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:25.343539 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dad9bb-07d1-4b40-9da6-8b5d3a7eb36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:25.344303 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dad9bb-07d1-4b40-9da6-8b5d3a7eb36b admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:25.344645 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dad9bb-07d1-4b40-9da6-8b5d3a7eb36b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:25.344920 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dad9bb-07d1-4b40-9da6-8b5d3a7eb36b admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:25.345569 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dad9bb-07d1-4b40-9da6-8b5d3a7eb36b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:25.347391 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dad9bb-07d1-4b40-9da6-8b5d3a7eb36b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:25.360200 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 825/1666] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:25 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:25.364597 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c3c7f1-0a8d-47b4-9091-1ab50914413b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:25.441369 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:25.505396 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c3c7f1-0a8d-47b4-9091-1ab50914413b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:25.511241 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb5fe57-7a65-4017-b4ba-91ed53f013fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:25.512098 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb5fe57-7a65-4017-b4ba-91ed53f013fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:25.512671 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb5fe57-7a65-4017-b4ba-91ed53f013fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:25.513163 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb5fe57-7a65-4017-b4ba-91ed53f013fb admin admin] PATH_INFO: `/v3/projects/d68cd6eb18ae469fa0e3167225d148e8/users/89fea20ab4bf4c2dbebb68496beb6c4b/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:25.515366 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb5fe57-7a65-4017-b4ba-91ed53f013fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d68cd6eb18ae469fa0e3167225d148e8, user_id=89fea20ab4bf4c2dbebb68496beb6c4b, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:25.518779 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb5fe57-7a65-4017-b4ba-91ed53f013fb admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:25.533681 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 842/1667] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:37:25 2022] PUT /identity/v3/projects/d68cd6eb18ae469fa0e3167225d148e8/users/89fea20ab4bf4c2dbebb68496beb6c4b/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:25.545447 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c104fa58-23a3-43ec-85aa-2e4294da439a None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:25.545715 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c104fa58-23a3-43ec-85aa-2e4294da439a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:25.545988 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c104fa58-23a3-43ec-85aa-2e4294da439a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:25.554809 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb5fe57-7a65-4017-b4ba-91ed53f013fb admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:25.556920 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1cb5fe57-7a65-4017-b4ba-91ed53f013fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:25.587119 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-c104fa58-23a3-43ec-85aa-2e4294da439a None None] MFA Rules not processed for user `89fea20ab4bf4c2dbebb68496beb6c4b`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:25.588639 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c104fa58-23a3-43ec-85aa-2e4294da439a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:25.630956 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb5fe57-7a65-4017-b4ba-91ed53f013fb admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:25.636067 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43622ae-1fa9-48ce-b0e8-ddc8a6495336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:25.636779 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43622ae-1fa9-48ce-b0e8-ddc8a6495336 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:25.637065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43622ae-1fa9-48ce-b0e8-ddc8a6495336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:25.637336 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43622ae-1fa9-48ce-b0e8-ddc8a6495336 admin admin] PATH_INFO: `/v3/projects/e33bba2fc97f45798bd2f9b22d0b4007` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:25.638474 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43622ae-1fa9-48ce-b0e8-ddc8a6495336 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e33bba2fc97f45798bd2f9b22d0b4007)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:25.640403 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43622ae-1fa9-48ce-b0e8-ddc8a6495336 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:25.661332 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 826/1668] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:25 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:25.675887 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c104fa58-23a3-43ec-85aa-2e4294da439a None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:25.677256 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c104fa58-23a3-43ec-85aa-2e4294da439a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:25.682144 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:25.691184 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f43622ae-1fa9-48ce-b0e8-ddc8a6495336 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e33bba2fc97f45798bd2f9b22d0b4007', 'request_id': 'req-f43622ae-1fa9-48ce-b0e8-ddc8a6495336'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:25.698742 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f43622ae-1fa9-48ce-b0e8-ddc8a6495336 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e33bba2fc97f45798bd2f9b22d0b4007', 'request_id': 'req-f43622ae-1fa9-48ce-b0e8-ddc8a6495336'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:25.708548 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f43622ae-1fa9-48ce-b0e8-ddc8a6495336 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e33bba2fc97f45798bd2f9b22d0b4007', 'request_id': 'req-f43622ae-1fa9-48ce-b0e8-ddc8a6495336'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:25.730200 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f43622ae-1fa9-48ce-b0e8-ddc8a6495336 admin admin] The token cache is being invalidate because project e33bba2fc97f45798bd2f9b22d0b4007 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:37:25.730547 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f43622ae-1fa9-48ce-b0e8-ddc8a6495336 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:25.731239 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f43622ae-1fa9-48ce-b0e8-ddc8a6495336 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:25.734113 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 843/1669] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:37:25 2022] DELETE /identity/v3/projects/e33bba2fc97f45798bd2f9b22d0b4007 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:37:25.739105 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823dcdf8-27af-4859-a6e6-7fea592d192d None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:25.739631 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823dcdf8-27af-4859-a6e6-7fea592d192d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:25.740172 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823dcdf8-27af-4859-a6e6-7fea592d192d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:25.747592 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c104fa58-23a3-43ec-85aa-2e4294da439a None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:25.769913 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-823dcdf8-27af-4859-a6e6-7fea592d192d None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:25.771544 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-823dcdf8-27af-4859-a6e6-7fea592d192d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:25.804737 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5096fecb-e839-4d8e-ad5f-4181e07378a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:25.805481 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096fecb-e839-4d8e-ad5f-4181e07378a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:25.805764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096fecb-e839-4d8e-ad5f-4181e07378a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:25.806038 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096fecb-e839-4d8e-ad5f-4181e07378a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:25.808803 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5096fecb-e839-4d8e-ad5f-4181e07378a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:25.824488 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:25.857567 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 844/1670] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:37:25.860348 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23dc76-4913-4c18-b18f-a87a24742581 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:25.860667 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23dc76-4913-4c18-b18f-a87a24742581 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:25.860950 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23dc76-4913-4c18-b18f-a87a24742581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:25.894989 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-3b23dc76-4913-4c18-b18f-a87a24742581 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:25.896188 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3b23dc76-4913-4c18-b18f-a87a24742581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:25.903256 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5096fecb-e839-4d8e-ad5f-4181e07378a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:25.904884 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5096fecb-e839-4d8e-ad5f-4181e07378a9 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:25.907526 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 845/1671] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:25.912211 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b23dc76-4913-4c18-b18f-a87a24742581 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:25.917341 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 827/1672] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:25 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:37:25.926618 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b23dc76-4913-4c18-b18f-a87a24742581 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:25.932327 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de9fc9-8b10-434f-a658-fab95561e879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:25.934033 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de9fc9-8b10-434f-a658-fab95561e879 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:25.934319 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de9fc9-8b10-434f-a658-fab95561e879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:25.934590 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de9fc9-8b10-434f-a658-fab95561e879 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:25.936044 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11de9fc9-8b10-434f-a658-fab95561e879 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:37:25.936355 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11de9fc9-8b10-434f-a658-fab95561e879 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:25.937977 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11de9fc9-8b10-434f-a658-fab95561e879 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:25.953957 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 846/1673] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:37:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:37:25.966028 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5096fecb-e839-4d8e-ad5f-4181e07378a9 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:25.978531 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5096fecb-e839-4d8e-ad5f-4181e07378a9 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:25.985562 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496bd3ce-95a4-445b-8c85-c51c6db7ea3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:25.986381 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bd3ce-95a4-445b-8c85-c51c6db7ea3d admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:25.986731 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bd3ce-95a4-445b-8c85-c51c6db7ea3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:25.987074 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bd3ce-95a4-445b-8c85-c51c6db7ea3d admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:25.987785 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496bd3ce-95a4-445b-8c85-c51c6db7ea3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:25.990109 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496bd3ce-95a4-445b-8c85-c51c6db7ea3d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:26.024723 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 828/1674] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:37:25 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:37:26.028694 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de9fc9-8b10-434f-a658-fab95561e879 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:26.044004 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de9fc9-8b10-434f-a658-fab95561e879 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:26.115784 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae317c3-1ab0-4f6c-8896-fa010875c29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:26.116401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae317c3-1ab0-4f6c-8896-fa010875c29d admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:26.116720 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae317c3-1ab0-4f6c-8896-fa010875c29d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:26.116993 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae317c3-1ab0-4f6c-8896-fa010875c29d admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:26.117627 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae317c3-1ab0-4f6c-8896-fa010875c29d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:26.120707 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae317c3-1ab0-4f6c-8896-fa010875c29d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:26.157329 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 847/1675] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:37:26 2022] POST /identity/v3/users => generated 287 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:37:26.161502 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496bd3ce-95a4-445b-8c85-c51c6db7ea3d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:26.176809 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496bd3ce-95a4-445b-8c85-c51c6db7ea3d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:26.182374 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c582946-1d9d-405f-b41a-59315995bbc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:26.182999 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c582946-1d9d-405f-b41a-59315995bbc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:26.183285 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c582946-1d9d-405f-b41a-59315995bbc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:26.183557 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c582946-1d9d-405f-b41a-59315995bbc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:26.184455 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c582946-1d9d-405f-b41a-59315995bbc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:26.187302 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c582946-1d9d-405f-b41a-59315995bbc7 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:26.206173 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 829/1676] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:26.210423 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae317c3-1ab0-4f6c-8896-fa010875c29d admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:26.222628 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae317c3-1ab0-4f6c-8896-fa010875c29d admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:26.228105 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c592671-9cbd-48c6-9313-e8eeec7ae90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:26.228989 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c592671-9cbd-48c6-9313-e8eeec7ae90c admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:26.229252 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c592671-9cbd-48c6-9313-e8eeec7ae90c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:26.229529 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c592671-9cbd-48c6-9313-e8eeec7ae90c admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:26.230164 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c592671-9cbd-48c6-9313-e8eeec7ae90c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:26.231767 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c592671-9cbd-48c6-9313-e8eeec7ae90c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:26.245040 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 848/1677] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:26.249905 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c582946-1d9d-405f-b41a-59315995bbc7 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:26.264634 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c582946-1d9d-405f-b41a-59315995bbc7 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:26.275097 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a0c61c-9d89-4b0f-a43a-da1749701b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:26.276319 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a0c61c-9d89-4b0f-a43a-da1749701b03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:26.277021 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a0c61c-9d89-4b0f-a43a-da1749701b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:26.277471 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a0c61c-9d89-4b0f-a43a-da1749701b03 admin admin] PATH_INFO: `/v3/projects/0e72ba25de034aa9970e0f31f2006a91/users/fa5e71cbbf8e46f585debc9530cff6b3/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:26.317793 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a0c61c-9d89-4b0f-a43a-da1749701b03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e72ba25de034aa9970e0f31f2006a91, user_id=fa5e71cbbf8e46f585debc9530cff6b3, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:26.321562 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a0c61c-9d89-4b0f-a43a-da1749701b03 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:26.361992 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 830/1678] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:37:26 2022] PUT /identity/v3/projects/0e72ba25de034aa9970e0f31f2006a91/users/fa5e71cbbf8e46f585debc9530cff6b3/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:26.366264 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfcdf6a-3b0e-4d99-b93c-f3fb6402af2a None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:26.366555 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfcdf6a-3b0e-4d99-b93c-f3fb6402af2a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:26.366829 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfcdf6a-3b0e-4d99-b93c-f3fb6402af2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:26.407360 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-dbfcdf6a-3b0e-4d99-b93c-f3fb6402af2a None None] MFA Rules not processed for user `fa5e71cbbf8e46f585debc9530cff6b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:26.408627 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-dbfcdf6a-3b0e-4d99-b93c-f3fb6402af2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:26.522747 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 849/1679] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:26.522863 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a0c61c-9d89-4b0f-a43a-da1749701b03 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:26.537738 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:26.599976 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a0c61c-9d89-4b0f-a43a-da1749701b03 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:26.605613 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aaa254-9ce6-4291-8b39-9d86fdfdf4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:26.606416 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aaa254-9ce6-4291-8b39-9d86fdfdf4fe admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:26.606862 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aaa254-9ce6-4291-8b39-9d86fdfdf4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:26.607831 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aaa254-9ce6-4291-8b39-9d86fdfdf4fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:26.608736 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aaa254-9ce6-4291-8b39-9d86fdfdf4fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:26.610865 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aaa254-9ce6-4291-8b39-9d86fdfdf4fe admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:26.646977 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 831/1680] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:37:26 2022] POST /identity/v3/projects => generated 375 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:37:26.651821 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfcdf6a-3b0e-4d99-b93c-f3fb6402af2a None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:26.665756 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfcdf6a-3b0e-4d99-b93c-f3fb6402af2a None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:26.704632 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:26.731320 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da88bfa3-f107-43ac-a275-1dd5a6cd9ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:26.731938 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da88bfa3-f107-43ac-a275-1dd5a6cd9ab3 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:26.732809 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da88bfa3-f107-43ac-a275-1dd5a6cd9ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:26.733082 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da88bfa3-f107-43ac-a275-1dd5a6cd9ab3 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:26.733712 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da88bfa3-f107-43ac-a275-1dd5a6cd9ab3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:26.735655 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da88bfa3-f107-43ac-a275-1dd5a6cd9ab3 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:26.760548 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 850/1681] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:37:26 2022] POST /identity/v3/users => generated 294 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:37:26.764873 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aaa254-9ce6-4291-8b39-9d86fdfdf4fe admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:26.775345 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da88bfa3-f107-43ac-a275-1dd5a6cd9ab3 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:26.777027 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-da88bfa3-f107-43ac-a275-1dd5a6cd9ab3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:26.777841 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aaa254-9ce6-4291-8b39-9d86fdfdf4fe admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:26.783059 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c60f8e-6862-4ec4-a372-fb371a1010ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:26.783660 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c60f8e-6862-4ec4-a372-fb371a1010ea admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:26.783937 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c60f8e-6862-4ec4-a372-fb371a1010ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:26.784366 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c60f8e-6862-4ec4-a372-fb371a1010ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:26.784870 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c60f8e-6862-4ec4-a372-fb371a1010ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:26.786459 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c60f8e-6862-4ec4-a372-fb371a1010ea admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:26.789526 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da88bfa3-f107-43ac-a275-1dd5a6cd9ab3 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:26.794131 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cb32b2-ca7c-4be1-aefe-115d6553c862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:26.794816 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb32b2-ca7c-4be1-aefe-115d6553c862 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:26.795098 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb32b2-ca7c-4be1-aefe-115d6553c862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:26.795364 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb32b2-ca7c-4be1-aefe-115d6553c862 admin admin] PATH_INFO: `/v3/users/fa6ee0920cc540a591cf4a7e5c150f98` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:26.796490 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cb32b2-ca7c-4be1-aefe-115d6553c862 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa6ee0920cc540a591cf4a7e5c150f98)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:26.797557 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 832/1682] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:26.798489 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cb32b2-ca7c-4be1-aefe-115d6553c862 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:26.801427 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c60f8e-6862-4ec4-a372-fb371a1010ea admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:26.811307 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c60f8e-6862-4ec4-a372-fb371a1010ea admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:26.815983 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903acdfa-6649-41ce-9e7b-e1e7f40bf4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:26.816639 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903acdfa-6649-41ce-9e7b-e1e7f40bf4c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:26.816915 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903acdfa-6649-41ce-9e7b-e1e7f40bf4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:26.817184 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903acdfa-6649-41ce-9e7b-e1e7f40bf4c4 admin admin] PATH_INFO: `/v3/projects/eb16f493efc141d4ae2b6effea24a3ae/users/9ec4fe0de7b44903aaffbd541ffdd055/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:26.833947 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903acdfa-6649-41ce-9e7b-e1e7f40bf4c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb16f493efc141d4ae2b6effea24a3ae, user_id=9ec4fe0de7b44903aaffbd541ffdd055, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:26.837438 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903acdfa-6649-41ce-9e7b-e1e7f40bf4c4 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:26.848650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e4cb32b2-ca7c-4be1-aefe-115d6553c862 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa6ee0920cc540a591cf4a7e5c150f98'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:26.851532 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 833/1683] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:37:26 2022] PUT /identity/v3/projects/eb16f493efc141d4ae2b6effea24a3ae/users/9ec4fe0de7b44903aaffbd541ffdd055/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:37:26.855349 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903acdfa-6649-41ce-9e7b-e1e7f40bf4c4 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:26.856686 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e4cb32b2-ca7c-4be1-aefe-115d6553c862 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa6ee0920cc540a591cf4a7e5c150f98'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:26.868732 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e4cb32b2-ca7c-4be1-aefe-115d6553c862 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa6ee0920cc540a591cf4a7e5c150f98'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:26.875896 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e4cb32b2-ca7c-4be1-aefe-115d6553c862 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa6ee0920cc540a591cf4a7e5c150f98'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:26.876589 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e4cb32b2-ca7c-4be1-aefe-115d6553c862 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa6ee0920cc540a591cf4a7e5c150f98'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:26.882909 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 851/1684] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:37:26 2022] DELETE /identity/v3/users/fa6ee0920cc540a591cf4a7e5c150f98 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:26.887401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dce3c0-ba3d-4a68-b86c-44e2341ce02c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:26.887763 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dce3c0-ba3d-4a68-b86c-44e2341ce02c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:26.888033 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dce3c0-ba3d-4a68-b86c-44e2341ce02c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:26.907789 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-96dce3c0-ba3d-4a68-b86c-44e2341ce02c None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:26.909087 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-96dce3c0-ba3d-4a68-b86c-44e2341ce02c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:26.918780 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903acdfa-6649-41ce-9e7b-e1e7f40bf4c4 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:26.978110 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 852/1685] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:37:26.984409 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dce3c0-ba3d-4a68-b86c-44e2341ce02c None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:26.984804 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375002da-cd4d-4ce6-adf2-af6e423c1979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:26.985476 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375002da-cd4d-4ce6-adf2-af6e423c1979 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:26.985807 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375002da-cd4d-4ce6-adf2-af6e423c1979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:26.985971 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-96dce3c0-ba3d-4a68-b86c-44e2341ce02c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:26.986066 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375002da-cd4d-4ce6-adf2-af6e423c1979 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:26.986765 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375002da-cd4d-4ce6-adf2-af6e423c1979 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:26.989455 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375002da-cd4d-4ce6-adf2-af6e423c1979 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:27.002737 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 834/1686] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 09:37:27.003867 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375002da-cd4d-4ce6-adf2-af6e423c1979 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:27.005605 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-375002da-cd4d-4ce6-adf2-af6e423c1979 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:27.015800 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:27.023275 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375002da-cd4d-4ce6-adf2-af6e423c1979 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:27.030583 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eed62b-0103-4a2c-849d-ed0afc6b3110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:27.032241 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eed62b-0103-4a2c-849d-ed0afc6b3110 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:27.033315 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eed62b-0103-4a2c-849d-ed0afc6b3110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:27.033601 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eed62b-0103-4a2c-849d-ed0afc6b3110 admin admin] PATH_INFO: `/v3/projects/eb16f493efc141d4ae2b6effea24a3ae/users/9ec4fe0de7b44903aaffbd541ffdd055/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:27.037437 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eed62b-0103-4a2c-849d-ed0afc6b3110 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb16f493efc141d4ae2b6effea24a3ae, user_id=9ec4fe0de7b44903aaffbd541ffdd055, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:27.037867 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:27.040408 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eed62b-0103-4a2c-849d-ed0afc6b3110 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:27.053450 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 835/1687] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:37:27 2022] PUT /identity/v3/projects/eb16f493efc141d4ae2b6effea24a3ae/users/9ec4fe0de7b44903aaffbd541ffdd055/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:37:27.057284 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dce3c0-ba3d-4a68-b86c-44e2341ce02c None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:27.063682 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0dcc2f-b052-4219-aec0-ef30741fd1a0 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:27.063981 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0dcc2f-b052-4219-aec0-ef30741fd1a0 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:27.064236 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0dcc2f-b052-4219-aec0-ef30741fd1a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:27.095756 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-df0dcc2f-b052-4219-aec0-ef30741fd1a0 None None] MFA Rules not processed for user `9ec4fe0de7b44903aaffbd541ffdd055`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:27.097156 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-df0dcc2f-b052-4219-aec0-ef30741fd1a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:27.115947 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8140ee2b-f336-4736-a5d4-2c18678123a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:27.116595 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140ee2b-f336-4736-a5d4-2c18678123a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:27.116875 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140ee2b-f336-4736-a5d4-2c18678123a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:27.117144 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140ee2b-f336-4736-a5d4-2c18678123a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:27.177015 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 836/1688] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:27 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:27.187917 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0dcc2f-b052-4219-aec0-ef30741fd1a0 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:27.189104 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8140ee2b-f336-4736-a5d4-2c18678123a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:27.189356 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-df0dcc2f-b052-4219-aec0-ef30741fd1a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:27.190541 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8140ee2b-f336-4736-a5d4-2c18678123a5 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:27.200294 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 853/1689] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 09:37:27.250155 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0dcc2f-b052-4219-aec0-ef30741fd1a0 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:27.254141 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21894b8d-6d1b-4a56-b22d-0ae5896fca89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:27.254745 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21894b8d-6d1b-4a56-b22d-0ae5896fca89 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:27.255020 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21894b8d-6d1b-4a56-b22d-0ae5896fca89 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:27.255295 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21894b8d-6d1b-4a56-b22d-0ae5896fca89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:27.257133 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-21894b8d-6d1b-4a56-b22d-0ae5896fca89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:27.344011 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21894b8d-6d1b-4a56-b22d-0ae5896fca89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:27.345859 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21894b8d-6d1b-4a56-b22d-0ae5896fca89 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:27.358304 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 837/1690] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 09:37:27.499904 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8140ee2b-f336-4736-a5d4-2c18678123a5 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:27.502846 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8140ee2b-f336-4736-a5d4-2c18678123a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:27.521481 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8140ee2b-f336-4736-a5d4-2c18678123a5 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:27.527226 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b2ba84-dc81-4ce5-8ee5-c3a691f1d796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:27.527865 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2ba84-dc81-4ce5-8ee5-c3a691f1d796 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:27.528139 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2ba84-dc81-4ce5-8ee5-c3a691f1d796 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:27.528402 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2ba84-dc81-4ce5-8ee5-c3a691f1d796 admin admin] PATH_INFO: `/v3/projects/c96d33b2dd654bd1bd2be63dac9682b7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:27.530194 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b2ba84-dc81-4ce5-8ee5-c3a691f1d796 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c96d33b2dd654bd1bd2be63dac9682b7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:27.532275 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b2ba84-dc81-4ce5-8ee5-c3a691f1d796 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:27.593426 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-49b2ba84-dc81-4ce5-8ee5-c3a691f1d796 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c96d33b2dd654bd1bd2be63dac9682b7', 'request_id': 'req-49b2ba84-dc81-4ce5-8ee5-c3a691f1d796'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:27.599267 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-49b2ba84-dc81-4ce5-8ee5-c3a691f1d796 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c96d33b2dd654bd1bd2be63dac9682b7', 'request_id': 'req-49b2ba84-dc81-4ce5-8ee5-c3a691f1d796'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:27.608751 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-49b2ba84-dc81-4ce5-8ee5-c3a691f1d796 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c96d33b2dd654bd1bd2be63dac9682b7', 'request_id': 'req-49b2ba84-dc81-4ce5-8ee5-c3a691f1d796'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:27.621046 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-49b2ba84-dc81-4ce5-8ee5-c3a691f1d796 admin admin] The token cache is being invalidate because project c96d33b2dd654bd1bd2be63dac9682b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:37:27.621333 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-49b2ba84-dc81-4ce5-8ee5-c3a691f1d796 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:27.621969 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-49b2ba84-dc81-4ce5-8ee5-c3a691f1d796 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:27.623543 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 854/1691] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:37:27 2022] DELETE /identity/v3/projects/c96d33b2dd654bd1bd2be63dac9682b7 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:37:27.627704 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21894b8d-6d1b-4a56-b22d-0ae5896fca89 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:27.639761 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21894b8d-6d1b-4a56-b22d-0ae5896fca89 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:27.674831 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:27.721263 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcde4fe-efe5-4e2b-a069-1c82dd2914df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:27.721912 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcde4fe-efe5-4e2b-a069-1c82dd2914df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:27.722193 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcde4fe-efe5-4e2b-a069-1c82dd2914df admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:27.722466 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcde4fe-efe5-4e2b-a069-1c82dd2914df admin admin] PATH_INFO: `/v3/users/89fea20ab4bf4c2dbebb68496beb6c4b` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:27.723665 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcde4fe-efe5-4e2b-a069-1c82dd2914df admin admin] RBAC: Authorizing `identity:delete_user(user_id=89fea20ab4bf4c2dbebb68496beb6c4b)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:27.725654 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcde4fe-efe5-4e2b-a069-1c82dd2914df admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:27.784025 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-cdcde4fe-efe5-4e2b-a069-1c82dd2914df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89fea20ab4bf4c2dbebb68496beb6c4b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:27.791163 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-cdcde4fe-efe5-4e2b-a069-1c82dd2914df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89fea20ab4bf4c2dbebb68496beb6c4b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:27.804677 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-cdcde4fe-efe5-4e2b-a069-1c82dd2914df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89fea20ab4bf4c2dbebb68496beb6c4b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:27.813140 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-cdcde4fe-efe5-4e2b-a069-1c82dd2914df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89fea20ab4bf4c2dbebb68496beb6c4b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:27.814070 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-cdcde4fe-efe5-4e2b-a069-1c82dd2914df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89fea20ab4bf4c2dbebb68496beb6c4b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:27.820562 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 838/1692] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:37:27 2022] DELETE /identity/v3/users/89fea20ab4bf4c2dbebb68496beb6c4b => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:37:28.097720 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b2ba84-dc81-4ce5-8ee5-c3a691f1d796 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:28.099331 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-49b2ba84-dc81-4ce5-8ee5-c3a691f1d796 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:28.112824 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b2ba84-dc81-4ce5-8ee5-c3a691f1d796 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:28.139095 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:28.173392 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0588de5-f1cd-4d0f-ae27-f4b36681b477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:28.174077 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0588de5-f1cd-4d0f-ae27-f4b36681b477 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:28.174352 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0588de5-f1cd-4d0f-ae27-f4b36681b477 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:28.174626 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0588de5-f1cd-4d0f-ae27-f4b36681b477 admin admin] PATH_INFO: `/v3/projects/d68cd6eb18ae469fa0e3167225d148e8` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:28.175749 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0588de5-f1cd-4d0f-ae27-f4b36681b477 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d68cd6eb18ae469fa0e3167225d148e8)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:28.177667 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0588de5-f1cd-4d0f-ae27-f4b36681b477 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:28.223724 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e0588de5-f1cd-4d0f-ae27-f4b36681b477 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd68cd6eb18ae469fa0e3167225d148e8', 'request_id': 'req-e0588de5-f1cd-4d0f-ae27-f4b36681b477'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:28.229182 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e0588de5-f1cd-4d0f-ae27-f4b36681b477 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd68cd6eb18ae469fa0e3167225d148e8', 'request_id': 'req-e0588de5-f1cd-4d0f-ae27-f4b36681b477'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:28.237469 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e0588de5-f1cd-4d0f-ae27-f4b36681b477 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd68cd6eb18ae469fa0e3167225d148e8', 'request_id': 'req-e0588de5-f1cd-4d0f-ae27-f4b36681b477'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:28.249097 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e0588de5-f1cd-4d0f-ae27-f4b36681b477 admin admin] The token cache is being invalidate because project d68cd6eb18ae469fa0e3167225d148e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:37:28.249414 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e0588de5-f1cd-4d0f-ae27-f4b36681b477 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:28.253275 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e0588de5-f1cd-4d0f-ae27-f4b36681b477 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:28.254779 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 855/1693] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:37:28 2022] DELETE /identity/v3/projects/d68cd6eb18ae469fa0e3167225d148e8 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:28.259607 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd011d-3b62-4d1b-b3ab-245ca818f007 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:28.259969 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd011d-3b62-4d1b-b3ab-245ca818f007 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:28.260306 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd011d-3b62-4d1b-b3ab-245ca818f007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:28.284503 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-f2cd011d-3b62-4d1b-b3ab-245ca818f007 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:28.286145 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f2cd011d-3b62-4d1b-b3ab-245ca818f007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:28.346011 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 839/1694] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:37:28.353675 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1721f317-7ff6-4f6b-868e-c09c3620a487 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:28.353953 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1721f317-7ff6-4f6b-868e-c09c3620a487 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:28.354223 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1721f317-7ff6-4f6b-868e-c09c3620a487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:28.374472 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-1721f317-7ff6-4f6b-868e-c09c3620a487 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:28.375659 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1721f317-7ff6-4f6b-868e-c09c3620a487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:28.383356 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 856/1695] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:28.387598 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cd011d-3b62-4d1b-b3ab-245ca818f007 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:28.397255 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cd011d-3b62-4d1b-b3ab-245ca818f007 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:28.401832 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45724840-79ed-46e7-9279-f5f381ade3c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:28.402426 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45724840-79ed-46e7-9279-f5f381ade3c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:28.402703 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45724840-79ed-46e7-9279-f5f381ade3c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:28.402971 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45724840-79ed-46e7-9279-f5f381ade3c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:28.403605 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45724840-79ed-46e7-9279-f5f381ade3c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:37:28.403904 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45724840-79ed-46e7-9279-f5f381ade3c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:28.405528 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45724840-79ed-46e7-9279-f5f381ade3c9 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:28.416855 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 840/1696] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:37:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:37:28.421383 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1721f317-7ff6-4f6b-868e-c09c3620a487 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:28.430786 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1721f317-7ff6-4f6b-868e-c09c3620a487 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:28.435851 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90025d20-a618-4ac5-8b5a-0fc67df150d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:28.436439 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90025d20-a618-4ac5-8b5a-0fc67df150d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:28.436739 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90025d20-a618-4ac5-8b5a-0fc67df150d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:28.437005 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90025d20-a618-4ac5-8b5a-0fc67df150d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:28.437632 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90025d20-a618-4ac5-8b5a-0fc67df150d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:28.439503 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90025d20-a618-4ac5-8b5a-0fc67df150d2 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:28.474965 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 857/1697] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:37:28 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:37:28.477133 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45724840-79ed-46e7-9279-f5f381ade3c9 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:28.503763 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45724840-79ed-46e7-9279-f5f381ade3c9 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:28.570342 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:28.606407 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c1366-0114-402d-b8ed-011d1c42bc45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:28.608011 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c1366-0114-402d-b8ed-011d1c42bc45 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:28.608011 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c1366-0114-402d-b8ed-011d1c42bc45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:28.608335 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c1366-0114-402d-b8ed-011d1c42bc45 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:28.609023 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c1366-0114-402d-b8ed-011d1c42bc45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:28.610986 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c1366-0114-402d-b8ed-011d1c42bc45 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:28.645011 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 841/1698] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:37:28 2022] POST /identity/v3/users => generated 292 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:37:28.649160 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90025d20-a618-4ac5-8b5a-0fc67df150d2 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:28.662375 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90025d20-a618-4ac5-8b5a-0fc67df150d2 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:28.667743 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf1806-e549-45b6-aaa3-d778e3b8c753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:28.668352 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf1806-e549-45b6-aaa3-d778e3b8c753 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:28.668682 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf1806-e549-45b6-aaa3-d778e3b8c753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:28.668951 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf1806-e549-45b6-aaa3-d778e3b8c753 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:28.670510 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bf1806-e549-45b6-aaa3-d778e3b8c753 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:28.672095 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bf1806-e549-45b6-aaa3-d778e3b8c753 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:28.682857 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 858/1699] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:28.687413 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c1366-0114-402d-b8ed-011d1c42bc45 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:28.697583 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c1366-0114-402d-b8ed-011d1c42bc45 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:28.703508 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d77812-fbda-4212-a057-c618065d03c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:28.704104 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d77812-fbda-4212-a057-c618065d03c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:28.704379 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d77812-fbda-4212-a057-c618065d03c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:28.704689 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d77812-fbda-4212-a057-c618065d03c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:28.705570 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d77812-fbda-4212-a057-c618065d03c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:28.707139 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d77812-fbda-4212-a057-c618065d03c8 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:28.717444 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 842/1700] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:28.722073 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf1806-e549-45b6-aaa3-d778e3b8c753 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:28.732108 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf1806-e549-45b6-aaa3-d778e3b8c753 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:28.737276 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d6ab8c-a31b-4f56-abf2-c42acfff6db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:28.737983 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6ab8c-a31b-4f56-abf2-c42acfff6db9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:28.738212 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6ab8c-a31b-4f56-abf2-c42acfff6db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:28.738465 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6ab8c-a31b-4f56-abf2-c42acfff6db9 admin admin] PATH_INFO: `/v3/projects/80b6cb3d42684a2eacbbbe9a2550c283/users/258fa9c3996044958a6ed86698f4dc50/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:28.764702 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d6ab8c-a31b-4f56-abf2-c42acfff6db9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80b6cb3d42684a2eacbbbe9a2550c283, user_id=258fa9c3996044958a6ed86698f4dc50, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:28.764702 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d6ab8c-a31b-4f56-abf2-c42acfff6db9 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:28.770702 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 859/1701] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:37:28 2022] PUT /identity/v3/projects/80b6cb3d42684a2eacbbbe9a2550c283/users/258fa9c3996044958a6ed86698f4dc50/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:28.781525 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef49f72c-ed78-4bed-be04-ff0773a4b3bf None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:28.781810 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef49f72c-ed78-4bed-be04-ff0773a4b3bf None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:28.782082 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef49f72c-ed78-4bed-be04-ff0773a4b3bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:28.809946 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-ef49f72c-ed78-4bed-be04-ff0773a4b3bf None None] MFA Rules not processed for user `258fa9c3996044958a6ed86698f4dc50`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:28.811645 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ef49f72c-ed78-4bed-be04-ff0773a4b3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:28.878581 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 843/1702] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:28.883150 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d6ab8c-a31b-4f56-abf2-c42acfff6db9 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:28.952224 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:28.959183 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d6ab8c-a31b-4f56-abf2-c42acfff6db9 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:28.977210 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be88a34-db78-4410-a609-2a173cb5245b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:28.977855 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be88a34-db78-4410-a609-2a173cb5245b admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:28.979327 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be88a34-db78-4410-a609-2a173cb5245b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:28.979607 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be88a34-db78-4410-a609-2a173cb5245b admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:28.980310 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be88a34-db78-4410-a609-2a173cb5245b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:28.983643 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be88a34-db78-4410-a609-2a173cb5245b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:29.048890 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef49f72c-ed78-4bed-be04-ff0773a4b3bf None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:29.049099 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 860/1703] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:37:28 2022] POST /identity/v3/projects => generated 383 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:37:29.062526 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef49f72c-ed78-4bed-be04-ff0773a4b3bf None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:29.148472 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae72913-a80b-458b-9ec4-693affdef73d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:29.149540 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae72913-a80b-458b-9ec4-693affdef73d admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:29.149540 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae72913-a80b-458b-9ec4-693affdef73d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:29.149801 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae72913-a80b-458b-9ec4-693affdef73d admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:29.150813 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae72913-a80b-458b-9ec4-693affdef73d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:29.152936 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae72913-a80b-458b-9ec4-693affdef73d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:29.179304 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 844/1704] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:37:29 2022] POST /identity/v3/users => generated 298 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:37:29.183188 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be88a34-db78-4410-a609-2a173cb5245b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:29.193552 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be88a34-db78-4410-a609-2a173cb5245b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:29.198237 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f37b83-8506-4c9c-bcbb-8783d0457e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:29.198907 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f37b83-8506-4c9c-bcbb-8783d0457e86 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:29.199247 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f37b83-8506-4c9c-bcbb-8783d0457e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:29.199577 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f37b83-8506-4c9c-bcbb-8783d0457e86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:29.200269 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f37b83-8506-4c9c-bcbb-8783d0457e86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:29.202490 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f37b83-8506-4c9c-bcbb-8783d0457e86 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:29.213523 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 861/1705] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:29.218447 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae72913-a80b-458b-9ec4-693affdef73d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:29.228551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae72913-a80b-458b-9ec4-693affdef73d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:29.233521 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0208a466-9aa9-4945-9d57-b056da44dfa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:29.234188 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208a466-9aa9-4945-9d57-b056da44dfa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:29.234493 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208a466-9aa9-4945-9d57-b056da44dfa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:29.240447 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208a466-9aa9-4945-9d57-b056da44dfa1 admin admin] PATH_INFO: `/v3/projects/b3875b0182ea49669ecf029579f96658/users/29b372d87a694c6f83a075887a4e0cba/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:29.260669 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0208a466-9aa9-4945-9d57-b056da44dfa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3875b0182ea49669ecf029579f96658, user_id=29b372d87a694c6f83a075887a4e0cba, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:29.264005 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0208a466-9aa9-4945-9d57-b056da44dfa1 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:29.291872 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 845/1706] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:37:29 2022] PUT /identity/v3/projects/b3875b0182ea49669ecf029579f96658/users/29b372d87a694c6f83a075887a4e0cba/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:29.295785 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f37b83-8506-4c9c-bcbb-8783d0457e86 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:29.305643 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f37b83-8506-4c9c-bcbb-8783d0457e86 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:29.361203 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1334e22-de99-4653-ae83-555509d42d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:29.361841 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1334e22-de99-4653-ae83-555509d42d9f admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:29.362120 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1334e22-de99-4653-ae83-555509d42d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:29.362386 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1334e22-de99-4653-ae83-555509d42d9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:29.363018 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1334e22-de99-4653-ae83-555509d42d9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:29.364611 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1334e22-de99-4653-ae83-555509d42d9f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:29.377961 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 862/1707] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:29 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:29.379114 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0208a466-9aa9-4945-9d57-b056da44dfa1 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:29.389923 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0208a466-9aa9-4945-9d57-b056da44dfa1 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:29.394544 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae730f6-40fd-461c-b432-1c7926dd448b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:29.395203 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae730f6-40fd-461c-b432-1c7926dd448b admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:29.395482 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae730f6-40fd-461c-b432-1c7926dd448b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:29.395748 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae730f6-40fd-461c-b432-1c7926dd448b admin admin] PATH_INFO: `/v3/projects/b3875b0182ea49669ecf029579f96658/users/29b372d87a694c6f83a075887a4e0cba/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:29.397673 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae730f6-40fd-461c-b432-1c7926dd448b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3875b0182ea49669ecf029579f96658, user_id=29b372d87a694c6f83a075887a4e0cba, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:29.401256 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae730f6-40fd-461c-b432-1c7926dd448b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:29.406447 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:29.412610 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 846/1708] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:37:29 2022] PUT /identity/v3/projects/b3875b0182ea49669ecf029579f96658/users/29b372d87a694c6f83a075887a4e0cba/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:29.424455 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04496c1-8b47-428c-a9de-b785e9c770be None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:29.424772 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04496c1-8b47-428c-a9de-b785e9c770be None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:29.425054 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04496c1-8b47-428c-a9de-b785e9c770be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:29.456484 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-c04496c1-8b47-428c-a9de-b785e9c770be None None] MFA Rules not processed for user `29b372d87a694c6f83a075887a4e0cba`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:29.457741 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c04496c1-8b47-428c-a9de-b785e9c770be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:29.535234 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 863/1709] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:29 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:29.546565 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae730f6-40fd-461c-b432-1c7926dd448b admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:29.548282 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-eae730f6-40fd-461c-b432-1c7926dd448b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:29.605469 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae730f6-40fd-461c-b432-1c7926dd448b admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:29.609064 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912c68f6-97dc-49a1-b886-944de8a63d68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:29.609658 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c68f6-97dc-49a1-b886-944de8a63d68 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:29.609936 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c68f6-97dc-49a1-b886-944de8a63d68 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:29.610200 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c68f6-97dc-49a1-b886-944de8a63d68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:29.619358 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912c68f6-97dc-49a1-b886-944de8a63d68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:29.620802 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912c68f6-97dc-49a1-b886-944de8a63d68 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:29.631147 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 847/1710] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:29 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:37:29.811542 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04496c1-8b47-428c-a9de-b785e9c770be None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:29.813809 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c04496c1-8b47-428c-a9de-b785e9c770be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:29.827788 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04496c1-8b47-428c-a9de-b785e9c770be None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:29.832401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450f1b54-f5b9-4311-9b54-4f97c62cb612 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:29.833109 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f1b54-f5b9-4311-9b54-4f97c62cb612 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:29.833381 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f1b54-f5b9-4311-9b54-4f97c62cb612 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:29.833696 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f1b54-f5b9-4311-9b54-4f97c62cb612 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:29.895190 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:29.907378 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450f1b54-f5b9-4311-9b54-4f97c62cb612 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:29.908939 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450f1b54-f5b9-4311-9b54-4f97c62cb612 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:29.919264 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 864/1711] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:37:30.402884 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912c68f6-97dc-49a1-b886-944de8a63d68 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:30.511541 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912c68f6-97dc-49a1-b886-944de8a63d68 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:30.516776 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c6010f-599e-43b6-8c94-75d9c08056d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:30.517409 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c6010f-599e-43b6-8c94-75d9c08056d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:30.517686 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c6010f-599e-43b6-8c94-75d9c08056d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:30.518057 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c6010f-599e-43b6-8c94-75d9c08056d9 admin admin] PATH_INFO: `/v3/users/258fa9c3996044958a6ed86698f4dc50` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:30.519207 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c6010f-599e-43b6-8c94-75d9c08056d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=258fa9c3996044958a6ed86698f4dc50)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:30.521326 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c6010f-599e-43b6-8c94-75d9c08056d9 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:30.578518 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a6c6010f-599e-43b6-8c94-75d9c08056d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '258fa9c3996044958a6ed86698f4dc50'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:30.586090 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a6c6010f-599e-43b6-8c94-75d9c08056d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '258fa9c3996044958a6ed86698f4dc50'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:30.598512 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a6c6010f-599e-43b6-8c94-75d9c08056d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '258fa9c3996044958a6ed86698f4dc50'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:30.605701 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a6c6010f-599e-43b6-8c94-75d9c08056d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '258fa9c3996044958a6ed86698f4dc50'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:30.606302 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a6c6010f-599e-43b6-8c94-75d9c08056d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '258fa9c3996044958a6ed86698f4dc50'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:30.612603 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 848/1712] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:37:30 2022] DELETE /identity/v3/users/258fa9c3996044958a6ed86698f4dc50 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:30.617109 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95042d9b-affd-489b-b4da-c22cdc204f7d None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:30.617563 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95042d9b-affd-489b-b4da-c22cdc204f7d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:30.617682 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95042d9b-affd-489b-b4da-c22cdc204f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:30.636546 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-95042d9b-affd-489b-b4da-c22cdc204f7d None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:30.637749 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-95042d9b-affd-489b-b4da-c22cdc204f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:30.697311 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 865/1713] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:37:30.710789 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c6010f-599e-43b6-8c94-75d9c08056d9 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:30.712310 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a6c6010f-599e-43b6-8c94-75d9c08056d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:30.736369 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:30.775912 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c6010f-599e-43b6-8c94-75d9c08056d9 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:30.780237 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8422e94-4a55-4485-906c-8540d4daee94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:30.780873 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8422e94-4a55-4485-906c-8540d4daee94 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:30.781148 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8422e94-4a55-4485-906c-8540d4daee94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:30.781416 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8422e94-4a55-4485-906c-8540d4daee94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:30.790322 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8422e94-4a55-4485-906c-8540d4daee94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:30.791823 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8422e94-4a55-4485-906c-8540d4daee94 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:30.801352 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 849/1714] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:37:31.080773 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95042d9b-affd-489b-b4da-c22cdc204f7d None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:31.082238 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-95042d9b-affd-489b-b4da-c22cdc204f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:31.101536 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95042d9b-affd-489b-b4da-c22cdc204f7d None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:31.110520 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55126962-d9f5-4d5c-9933-9c2e464f56c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:31.111162 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55126962-d9f5-4d5c-9933-9c2e464f56c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:31.111966 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55126962-d9f5-4d5c-9933-9c2e464f56c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:31.111966 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55126962-d9f5-4d5c-9933-9c2e464f56c5 admin admin] PATH_INFO: `/v3/projects/80b6cb3d42684a2eacbbbe9a2550c283` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:31.112920 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55126962-d9f5-4d5c-9933-9c2e464f56c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80b6cb3d42684a2eacbbbe9a2550c283)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:31.115112 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55126962-d9f5-4d5c-9933-9c2e464f56c5 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:31.172790 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-55126962-d9f5-4d5c-9933-9c2e464f56c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80b6cb3d42684a2eacbbbe9a2550c283', 'request_id': 'req-55126962-d9f5-4d5c-9933-9c2e464f56c5'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:31.178928 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-55126962-d9f5-4d5c-9933-9c2e464f56c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80b6cb3d42684a2eacbbbe9a2550c283', 'request_id': 'req-55126962-d9f5-4d5c-9933-9c2e464f56c5'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:31.187594 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-55126962-d9f5-4d5c-9933-9c2e464f56c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80b6cb3d42684a2eacbbbe9a2550c283', 'request_id': 'req-55126962-d9f5-4d5c-9933-9c2e464f56c5'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:31.189636 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8422e94-4a55-4485-906c-8540d4daee94 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:31.191058 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e8422e94-4a55-4485-906c-8540d4daee94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:31.202506 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-55126962-d9f5-4d5c-9933-9c2e464f56c5 admin admin] The token cache is being invalidate because project 80b6cb3d42684a2eacbbbe9a2550c283 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:37:31.202899 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-55126962-d9f5-4d5c-9933-9c2e464f56c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:31.203563 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-55126962-d9f5-4d5c-9933-9c2e464f56c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:31.205434 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 866/1715] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:37:31 2022] DELETE /identity/v3/projects/80b6cb3d42684a2eacbbbe9a2550c283 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:31.209144 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55126962-d9f5-4d5c-9933-9c2e464f56c5 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:31.210542 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-55126962-d9f5-4d5c-9933-9c2e464f56c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:31.278372 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8422e94-4a55-4485-906c-8540d4daee94 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:31.290802 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55126962-d9f5-4d5c-9933-9c2e464f56c5 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:31.297272 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1526b20-e3e8-45e2-a097-a703612a4507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:31.297909 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1526b20-e3e8-45e2-a097-a703612a4507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:31.298188 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1526b20-e3e8-45e2-a097-a703612a4507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:31.298442 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1526b20-e3e8-45e2-a097-a703612a4507 admin admin] PATH_INFO: `/v3/users/29b372d87a694c6f83a075887a4e0cba` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:31.299691 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1526b20-e3e8-45e2-a097-a703612a4507 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29b372d87a694c6f83a075887a4e0cba)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:31.301662 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1526b20-e3e8-45e2-a097-a703612a4507 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:31.318379 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:31.334998 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8768f332-7381-4d87-854a-47d21b627139 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:31.335617 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8768f332-7381-4d87-854a-47d21b627139 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:31.335975 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8768f332-7381-4d87-854a-47d21b627139 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:31.336249 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8768f332-7381-4d87-854a-47d21b627139 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:31.338402 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8768f332-7381-4d87-854a-47d21b627139 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:31.360080 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b1526b20-e3e8-45e2-a097-a703612a4507 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29b372d87a694c6f83a075887a4e0cba'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:31.367511 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b1526b20-e3e8-45e2-a097-a703612a4507 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29b372d87a694c6f83a075887a4e0cba'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:31.380368 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b1526b20-e3e8-45e2-a097-a703612a4507 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29b372d87a694c6f83a075887a4e0cba'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:31.388601 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b1526b20-e3e8-45e2-a097-a703612a4507 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29b372d87a694c6f83a075887a4e0cba'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:31.389440 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b1526b20-e3e8-45e2-a097-a703612a4507 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29b372d87a694c6f83a075887a4e0cba'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:31.395591 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 867/1716] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:37:31 2022] DELETE /identity/v3/users/29b372d87a694c6f83a075887a4e0cba => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:37:31.396511 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1526b20-e3e8-45e2-a097-a703612a4507 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:31.397849 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b1526b20-e3e8-45e2-a097-a703612a4507 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:31.412231 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8768f332-7381-4d87-854a-47d21b627139 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:31.414181 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8768f332-7381-4d87-854a-47d21b627139 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:31.427559 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 850/1717] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:31 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 09:37:31.488244 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1526b20-e3e8-45e2-a097-a703612a4507 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:31.493667 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b44763-0647-45a1-ab74-2ba879d1fb1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:31.494300 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b44763-0647-45a1-ab74-2ba879d1fb1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:31.494691 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b44763-0647-45a1-ab74-2ba879d1fb1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:31.494884 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b44763-0647-45a1-ab74-2ba879d1fb1a admin admin] PATH_INFO: `/v3/users/fa5e71cbbf8e46f585debc9530cff6b3` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:31.496056 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b44763-0647-45a1-ab74-2ba879d1fb1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa5e71cbbf8e46f585debc9530cff6b3)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:31.498822 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b44763-0647-45a1-ab74-2ba879d1fb1a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:31.572726 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-20b44763-0647-45a1-ab74-2ba879d1fb1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa5e71cbbf8e46f585debc9530cff6b3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:31.586120 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-20b44763-0647-45a1-ab74-2ba879d1fb1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa5e71cbbf8e46f585debc9530cff6b3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:31.607399 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-20b44763-0647-45a1-ab74-2ba879d1fb1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa5e71cbbf8e46f585debc9530cff6b3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:31.621528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-20b44763-0647-45a1-ab74-2ba879d1fb1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa5e71cbbf8e46f585debc9530cff6b3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:31.623681 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-20b44763-0647-45a1-ab74-2ba879d1fb1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa5e71cbbf8e46f585debc9530cff6b3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:31.632890 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 868/1718] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:37:31 2022] DELETE /identity/v3/users/fa5e71cbbf8e46f585debc9530cff6b3 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:31.639477 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b9f0b-cc6a-4aae-9477-08b1eff7fc5e None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:31.639769 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b9f0b-cc6a-4aae-9477-08b1eff7fc5e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:31.640214 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b9f0b-cc6a-4aae-9477-08b1eff7fc5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:31.673536 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-877b9f0b-cc6a-4aae-9477-08b1eff7fc5e None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:31.687245 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-877b9f0b-cc6a-4aae-9477-08b1eff7fc5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:31.767180 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:31.863484 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 851/1719] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:31.877309 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b44763-0647-45a1-ab74-2ba879d1fb1a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:31.877309 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-20b44763-0647-45a1-ab74-2ba879d1fb1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:31.879183 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877b9f0b-cc6a-4aae-9477-08b1eff7fc5e None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:31.881117 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-877b9f0b-cc6a-4aae-9477-08b1eff7fc5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:31.902395 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877b9f0b-cc6a-4aae-9477-08b1eff7fc5e None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:31.903575 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:31.914228 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aee501-85e4-420a-a7c7-4e063407d758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:31.926994 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aee501-85e4-420a-a7c7-4e063407d758 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:31.928907 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aee501-85e4-420a-a7c7-4e063407d758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:31.932329 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aee501-85e4-420a-a7c7-4e063407d758 admin admin] PATH_INFO: `/v3/projects/b3875b0182ea49669ecf029579f96658` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:31.936647 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aee501-85e4-420a-a7c7-4e063407d758 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3875b0182ea49669ecf029579f96658)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:31.939373 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aee501-85e4-420a-a7c7-4e063407d758 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:32.046671 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f3aee501-85e4-420a-a7c7-4e063407d758 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3875b0182ea49669ecf029579f96658', 'request_id': 'req-f3aee501-85e4-420a-a7c7-4e063407d758'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:32.053539 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b44763-0647-45a1-ab74-2ba879d1fb1a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:32.055768 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f3aee501-85e4-420a-a7c7-4e063407d758 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3875b0182ea49669ecf029579f96658', 'request_id': 'req-f3aee501-85e4-420a-a7c7-4e063407d758'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:32.062682 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f3aee501-85e4-420a-a7c7-4e063407d758 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3875b0182ea49669ecf029579f96658', 'request_id': 'req-f3aee501-85e4-420a-a7c7-4e063407d758'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:32.090652 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f3aee501-85e4-420a-a7c7-4e063407d758 admin admin] The token cache is being invalidate because project b3875b0182ea49669ecf029579f96658 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:37:32.090953 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f3aee501-85e4-420a-a7c7-4e063407d758 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:32.093758 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f3aee501-85e4-420a-a7c7-4e063407d758 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:32.097898 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 852/1720] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:37:31 2022] DELETE /identity/v3/projects/b3875b0182ea49669ecf029579f96658 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:32.105014 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa52ec3-05cd-4005-affa-3de1aa0b583b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:32.105306 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa52ec3-05cd-4005-affa-3de1aa0b583b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:32.105882 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa52ec3-05cd-4005-affa-3de1aa0b583b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:32.137048 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-0aa52ec3-05cd-4005-affa-3de1aa0b583b None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:32.139642 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0aa52ec3-05cd-4005-affa-3de1aa0b583b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:32.157657 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c531f-9531-4002-8096-e8c80d677e44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:32.157657 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c531f-9531-4002-8096-e8c80d677e44 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:32.157657 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c531f-9531-4002-8096-e8c80d677e44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:32.157657 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c531f-9531-4002-8096-e8c80d677e44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:32.160265 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-522c531f-9531-4002-8096-e8c80d677e44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:32.243103 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 853/1721] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:32.251525 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2805de3-751d-4c03-9f45-8e6ab671c003 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:32.253253 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2805de3-751d-4c03-9f45-8e6ab671c003 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:32.253253 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2805de3-751d-4c03-9f45-8e6ab671c003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:32.290334 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522c531f-9531-4002-8096-e8c80d677e44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:32.292221 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522c531f-9531-4002-8096-e8c80d677e44 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:32.300583 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-d2805de3-751d-4c03-9f45-8e6ab671c003 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:32.305861 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d2805de3-751d-4c03-9f45-8e6ab671c003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:32.310904 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 869/1722] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:37:32.334856 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 854/1723] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:32.336680 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c531f-9531-4002-8096-e8c80d677e44 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:32.351625 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c531f-9531-4002-8096-e8c80d677e44 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:32.367461 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb11cb1c-2442-4192-a6d9-eaa3125fd045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:32.368848 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11cb1c-2442-4192-a6d9-eaa3125fd045 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:32.370455 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11cb1c-2442-4192-a6d9-eaa3125fd045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:32.370847 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11cb1c-2442-4192-a6d9-eaa3125fd045 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:32.374190 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb11cb1c-2442-4192-a6d9-eaa3125fd045 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:37:32.375007 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb11cb1c-2442-4192-a6d9-eaa3125fd045 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:32.378056 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb11cb1c-2442-4192-a6d9-eaa3125fd045 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:32.414026 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 870/1724] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:37:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:37:32.416649 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2805de3-751d-4c03-9f45-8e6ab671c003 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:32.433187 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2805de3-751d-4c03-9f45-8e6ab671c003 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:32.450455 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b987e-105a-4f0c-9244-906cfc4a986f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:32.451737 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b987e-105a-4f0c-9244-906cfc4a986f admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:32.452120 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b987e-105a-4f0c-9244-906cfc4a986f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:32.452494 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b987e-105a-4f0c-9244-906cfc4a986f admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:32.453272 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388b987e-105a-4f0c-9244-906cfc4a986f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:32.458472 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388b987e-105a-4f0c-9244-906cfc4a986f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:32.529817 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 855/1725] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:37:32 2022] POST /identity/v3/projects => generated 375 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:37:32.530994 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb11cb1c-2442-4192-a6d9-eaa3125fd045 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:32.543843 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb11cb1c-2442-4192-a6d9-eaa3125fd045 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:32.643268 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b7087f-d1f8-466f-acd6-e1715ed6e27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:32.643977 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7087f-d1f8-466f-acd6-e1715ed6e27c admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:32.644455 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7087f-d1f8-466f-acd6-e1715ed6e27c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:32.644994 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7087f-d1f8-466f-acd6-e1715ed6e27c admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:32.645774 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b7087f-d1f8-466f-acd6-e1715ed6e27c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:32.647784 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b7087f-d1f8-466f-acd6-e1715ed6e27c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:32.658947 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:32.676593 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 871/1726] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:37:32 2022] POST /identity/v3/users => generated 288 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:37:32.677896 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b987e-105a-4f0c-9244-906cfc4a986f admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:32.691941 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b987e-105a-4f0c-9244-906cfc4a986f admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:32.698979 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4bc12c-caef-4365-a656-59db9f3459bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:32.700073 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4bc12c-caef-4365-a656-59db9f3459bd admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:32.700595 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4bc12c-caef-4365-a656-59db9f3459bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:32.701047 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4bc12c-caef-4365-a656-59db9f3459bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:32.701836 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4bc12c-caef-4365-a656-59db9f3459bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:32.703881 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4bc12c-caef-4365-a656-59db9f3459bd admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:32.715882 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 856/1727] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:32 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:32.721701 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b7087f-d1f8-466f-acd6-e1715ed6e27c admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:32.732725 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b7087f-d1f8-466f-acd6-e1715ed6e27c admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:32.739053 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b310f206-d8ff-4658-83aa-c06a512509cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:32.739670 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b310f206-d8ff-4658-83aa-c06a512509cf admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:32.739953 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b310f206-d8ff-4658-83aa-c06a512509cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:32.740227 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b310f206-d8ff-4658-83aa-c06a512509cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:32.740925 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b310f206-d8ff-4658-83aa-c06a512509cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:32.742717 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b310f206-d8ff-4658-83aa-c06a512509cf admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:32.757247 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 872/1728] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:32.761457 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4bc12c-caef-4365-a656-59db9f3459bd admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:32.773214 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4bc12c-caef-4365-a656-59db9f3459bd admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:32.778401 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b6c17-47ac-4290-907e-7963e4106d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:32.779808 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b6c17-47ac-4290-907e-7963e4106d0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:32.780241 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b6c17-47ac-4290-907e-7963e4106d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:32.780701 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b6c17-47ac-4290-907e-7963e4106d0d admin admin] PATH_INFO: `/v3/projects/ff934d8402614cca93a0c8a5e36a177f/users/23dc03b6ed374e2699c059cc7172b753/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:32.800276 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:32.806442 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525b6c17-47ac-4290-907e-7963e4106d0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff934d8402614cca93a0c8a5e36a177f, user_id=23dc03b6ed374e2699c059cc7172b753, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:32.815407 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525b6c17-47ac-4290-907e-7963e4106d0d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:32.843588 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 857/1729] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:37:32 2022] PUT /identity/v3/projects/ff934d8402614cca93a0c8a5e36a177f/users/23dc03b6ed374e2699c059cc7172b753/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:32.855598 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008c895-e4af-490a-ae3a-652619cc2f99 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:32.857270 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008c895-e4af-490a-ae3a-652619cc2f99 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:32.857775 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008c895-e4af-490a-ae3a-652619cc2f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:32.901168 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-0008c895-e4af-490a-ae3a-652619cc2f99 None None] MFA Rules not processed for user `23dc03b6ed374e2699c059cc7172b753`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:32.903223 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0008c895-e4af-490a-ae3a-652619cc2f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:32.995942 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 873/1730] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:33.099294 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b6c17-47ac-4290-907e-7963e4106d0d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:33.101843 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-525b6c17-47ac-4290-907e-7963e4106d0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:33.229478 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b6c17-47ac-4290-907e-7963e4106d0d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:33.235129 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210eca05-ef00-456f-b5ad-c98460e45418 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:33.235889 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210eca05-ef00-456f-b5ad-c98460e45418 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:33.236240 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210eca05-ef00-456f-b5ad-c98460e45418 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:33.236609 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210eca05-ef00-456f-b5ad-c98460e45418 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:33.249995 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0008c895-e4af-490a-ae3a-652619cc2f99 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:33.251416 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0008c895-e4af-490a-ae3a-652619cc2f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:33.261281 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210eca05-ef00-456f-b5ad-c98460e45418 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:33.264249 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210eca05-ef00-456f-b5ad-c98460e45418 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:33.307462 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 858/1731] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:37:33.377951 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0008c895-e4af-490a-ae3a-652619cc2f99 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:33.385409 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811955f8-d305-45f3-a4ae-0fd1bb59b227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:33.390477 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811955f8-d305-45f3-a4ae-0fd1bb59b227 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:33.390816 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811955f8-d305-45f3-a4ae-0fd1bb59b227 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:33.391136 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811955f8-d305-45f3-a4ae-0fd1bb59b227 admin admin] PATH_INFO: `/v3/projects/0e72ba25de034aa9970e0f31f2006a91` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:33.393097 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811955f8-d305-45f3-a4ae-0fd1bb59b227 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e72ba25de034aa9970e0f31f2006a91)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:33.399017 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811955f8-d305-45f3-a4ae-0fd1bb59b227 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:33.483911 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-811955f8-d305-45f3-a4ae-0fd1bb59b227 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e72ba25de034aa9970e0f31f2006a91', 'request_id': 'req-811955f8-d305-45f3-a4ae-0fd1bb59b227'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:33.496400 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-811955f8-d305-45f3-a4ae-0fd1bb59b227 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e72ba25de034aa9970e0f31f2006a91', 'request_id': 'req-811955f8-d305-45f3-a4ae-0fd1bb59b227'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:33.509465 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-811955f8-d305-45f3-a4ae-0fd1bb59b227 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e72ba25de034aa9970e0f31f2006a91', 'request_id': 'req-811955f8-d305-45f3-a4ae-0fd1bb59b227'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:33.525671 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-811955f8-d305-45f3-a4ae-0fd1bb59b227 admin admin] The token cache is being invalidate because project 0e72ba25de034aa9970e0f31f2006a91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:37:33.526108 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-811955f8-d305-45f3-a4ae-0fd1bb59b227 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:33.526915 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-811955f8-d305-45f3-a4ae-0fd1bb59b227 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:33.532069 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 874/1732] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:37:33 2022] DELETE /identity/v3/projects/0e72ba25de034aa9970e0f31f2006a91 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:33.533424 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210eca05-ef00-456f-b5ad-c98460e45418 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:33.548183 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210eca05-ef00-456f-b5ad-c98460e45418 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:33.658634 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2106b60-5ce6-4c73-a4ff-0cb770534b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:33.661160 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2106b60-5ce6-4c73-a4ff-0cb770534b2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:33.666060 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2106b60-5ce6-4c73-a4ff-0cb770534b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:33.666253 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2106b60-5ce6-4c73-a4ff-0cb770534b2b admin admin] PATH_INFO: `/v3/users/9ec4fe0de7b44903aaffbd541ffdd055` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:33.669096 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2106b60-5ce6-4c73-a4ff-0cb770534b2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ec4fe0de7b44903aaffbd541ffdd055)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:33.674793 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2106b60-5ce6-4c73-a4ff-0cb770534b2b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:33.789340 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b2106b60-5ce6-4c73-a4ff-0cb770534b2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ec4fe0de7b44903aaffbd541ffdd055'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:33.830007 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b2106b60-5ce6-4c73-a4ff-0cb770534b2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ec4fe0de7b44903aaffbd541ffdd055'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:33.885651 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b2106b60-5ce6-4c73-a4ff-0cb770534b2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ec4fe0de7b44903aaffbd541ffdd055'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:33.926158 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b2106b60-5ce6-4c73-a4ff-0cb770534b2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ec4fe0de7b44903aaffbd541ffdd055'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:33.927002 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b2106b60-5ce6-4c73-a4ff-0cb770534b2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ec4fe0de7b44903aaffbd541ffdd055'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:33.938250 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 859/1733] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:37:33 2022] DELETE /identity/v3/users/9ec4fe0de7b44903aaffbd541ffdd055 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:35.071530 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811955f8-d305-45f3-a4ae-0fd1bb59b227 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:35.073854 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-811955f8-d305-45f3-a4ae-0fd1bb59b227 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:35.085055 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:35.094170 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811955f8-d305-45f3-a4ae-0fd1bb59b227 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:35.213580 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c13583d-6048-4e67-aa1e-4e5529704c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:35.214449 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c13583d-6048-4e67-aa1e-4e5529704c60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:35.214727 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c13583d-6048-4e67-aa1e-4e5529704c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:35.215777 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c13583d-6048-4e67-aa1e-4e5529704c60 admin admin] PATH_INFO: `/v3/projects/eb16f493efc141d4ae2b6effea24a3ae` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:35.217748 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c13583d-6048-4e67-aa1e-4e5529704c60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb16f493efc141d4ae2b6effea24a3ae)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:35.219796 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c13583d-6048-4e67-aa1e-4e5529704c60 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:35.322388 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5c13583d-6048-4e67-aa1e-4e5529704c60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb16f493efc141d4ae2b6effea24a3ae', 'request_id': 'req-5c13583d-6048-4e67-aa1e-4e5529704c60'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:35.334436 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5c13583d-6048-4e67-aa1e-4e5529704c60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb16f493efc141d4ae2b6effea24a3ae', 'request_id': 'req-5c13583d-6048-4e67-aa1e-4e5529704c60'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:35.348870 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5c13583d-6048-4e67-aa1e-4e5529704c60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb16f493efc141d4ae2b6effea24a3ae', 'request_id': 'req-5c13583d-6048-4e67-aa1e-4e5529704c60'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:35.375534 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5c13583d-6048-4e67-aa1e-4e5529704c60 admin admin] The token cache is being invalidate because project eb16f493efc141d4ae2b6effea24a3ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:37:35.375831 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5c13583d-6048-4e67-aa1e-4e5529704c60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:35.376877 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5c13583d-6048-4e67-aa1e-4e5529704c60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:35.383157 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 875/1734] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:37:35 2022] DELETE /identity/v3/projects/eb16f493efc141d4ae2b6effea24a3ae => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:35.394793 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270b12e7-4301-4803-8d48-6cef862f59f0 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:35.395294 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270b12e7-4301-4803-8d48-6cef862f59f0 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:35.395388 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270b12e7-4301-4803-8d48-6cef862f59f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:35.469783 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-270b12e7-4301-4803-8d48-6cef862f59f0 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:35.472646 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-270b12e7-4301-4803-8d48-6cef862f59f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:35.499088 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:35.670708 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 860/1735] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:35.678742 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8d399-c8ba-454c-8219-16efeb5638d5 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:35.679030 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8d399-c8ba-454c-8219-16efeb5638d5 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:35.679294 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8d399-c8ba-454c-8219-16efeb5638d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:35.741813 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-e1c8d399-c8ba-454c-8219-16efeb5638d5 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:35.743063 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e1c8d399-c8ba-454c-8219-16efeb5638d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:35.770857 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 876/1736] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:35.774727 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270b12e7-4301-4803-8d48-6cef862f59f0 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:35.792581 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270b12e7-4301-4803-8d48-6cef862f59f0 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:35.807215 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f70bed-b747-4134-ad92-97c16a0f7fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:35.808359 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f70bed-b747-4134-ad92-97c16a0f7fc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:35.809153 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f70bed-b747-4134-ad92-97c16a0f7fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:35.810438 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f70bed-b747-4134-ad92-97c16a0f7fc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:35.811347 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f70bed-b747-4134-ad92-97c16a0f7fc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:37:35.811921 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f70bed-b747-4134-ad92-97c16a0f7fc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:35.813890 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f70bed-b747-4134-ad92-97c16a0f7fc8 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:35.831683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 861/1737] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:37:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:37:35.836937 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c8d399-c8ba-454c-8219-16efeb5638d5 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:35.849242 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c8d399-c8ba-454c-8219-16efeb5638d5 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:35.855529 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e5fcfb-e0fd-4293-8bde-bff455c86e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:35.856378 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5fcfb-e0fd-4293-8bde-bff455c86e58 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:35.857727 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5fcfb-e0fd-4293-8bde-bff455c86e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:35.858313 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5fcfb-e0fd-4293-8bde-bff455c86e58 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:35.859323 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e5fcfb-e0fd-4293-8bde-bff455c86e58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:35.863642 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e5fcfb-e0fd-4293-8bde-bff455c86e58 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:35.935382 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 877/1738] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:37:35 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:37:35.940461 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f70bed-b747-4134-ad92-97c16a0f7fc8 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:35.959410 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f70bed-b747-4134-ad92-97c16a0f7fc8 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:36.132219 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550bff2c-f80b-4472-8a07-00dae42e6fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:36.135383 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550bff2c-f80b-4472-8a07-00dae42e6fe9 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:36.135671 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550bff2c-f80b-4472-8a07-00dae42e6fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:36.135948 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550bff2c-f80b-4472-8a07-00dae42e6fe9 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:36.140138 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550bff2c-f80b-4472-8a07-00dae42e6fe9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:36.147552 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550bff2c-f80b-4472-8a07-00dae42e6fe9 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:36.202576 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 862/1739] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:37:35 2022] POST /identity/v3/users => generated 289 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:37:36.207328 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e5fcfb-e0fd-4293-8bde-bff455c86e58 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:36.225132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e5fcfb-e0fd-4293-8bde-bff455c86e58 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:36.236678 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087cb0c5-acd5-4a61-8c03-24124101b411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:36.237714 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087cb0c5-acd5-4a61-8c03-24124101b411 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:36.238390 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087cb0c5-acd5-4a61-8c03-24124101b411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:36.239244 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087cb0c5-acd5-4a61-8c03-24124101b411 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:36.240153 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087cb0c5-acd5-4a61-8c03-24124101b411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:36.242438 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087cb0c5-acd5-4a61-8c03-24124101b411 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:36.265155 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 878/1740] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:36 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:36.267203 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550bff2c-f80b-4472-8a07-00dae42e6fe9 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:36.293732 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550bff2c-f80b-4472-8a07-00dae42e6fe9 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:36.314726 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb407d-589a-4cfa-bc44-e9746a302c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:36.315759 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb407d-589a-4cfa-bc44-e9746a302c1b admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:36.316443 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb407d-589a-4cfa-bc44-e9746a302c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:36.317234 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb407d-589a-4cfa-bc44-e9746a302c1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:36.318077 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecb407d-589a-4cfa-bc44-e9746a302c1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:36.319924 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecb407d-589a-4cfa-bc44-e9746a302c1b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:36.335744 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 863/1741] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:36 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:36.342171 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087cb0c5-acd5-4a61-8c03-24124101b411 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:36.363235 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087cb0c5-acd5-4a61-8c03-24124101b411 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:36.376112 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86082a06-f2f0-43aa-b6c3-b26a059179e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:36.376834 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86082a06-f2f0-43aa-b6c3-b26a059179e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:36.377533 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86082a06-f2f0-43aa-b6c3-b26a059179e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:36.377533 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86082a06-f2f0-43aa-b6c3-b26a059179e6 admin admin] PATH_INFO: `/v3/projects/7d8a9ad4232a4413822b3ae6ea9613be/users/fa86d4d5527d470e98ca377204463e54/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:36.405500 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86082a06-f2f0-43aa-b6c3-b26a059179e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d8a9ad4232a4413822b3ae6ea9613be, user_id=fa86d4d5527d470e98ca377204463e54, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:36.408829 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86082a06-f2f0-43aa-b6c3-b26a059179e6 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:36.437185 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 879/1742] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:37:36 2022] PUT /identity/v3/projects/7d8a9ad4232a4413822b3ae6ea9613be/users/fa86d4d5527d470e98ca377204463e54/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:36.449981 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93549cf8-3b65-4ba6-af91-bc159cc746df None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:36.450486 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93549cf8-3b65-4ba6-af91-bc159cc746df None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:36.450486 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93549cf8-3b65-4ba6-af91-bc159cc746df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:36.505640 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-93549cf8-3b65-4ba6-af91-bc159cc746df None None] MFA Rules not processed for user `fa86d4d5527d470e98ca377204463e54`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:36.507087 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-93549cf8-3b65-4ba6-af91-bc159cc746df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:36.630684 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 864/1743] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:36.633984 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86082a06-f2f0-43aa-b6c3-b26a059179e6 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:36.690824 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:36.768635 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86082a06-f2f0-43aa-b6c3-b26a059179e6 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:36.776331 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf6944c-6906-472d-bf79-dee639834abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:36.777259 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf6944c-6906-472d-bf79-dee639834abe admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:36.778363 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf6944c-6906-472d-bf79-dee639834abe admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:36.779348 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf6944c-6906-472d-bf79-dee639834abe admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:36.780155 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf6944c-6906-472d-bf79-dee639834abe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:36.784016 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf6944c-6906-472d-bf79-dee639834abe admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:36.826713 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 880/1744] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:37:36 2022] POST /identity/v3/projects => generated 377 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:37:36.827883 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93549cf8-3b65-4ba6-af91-bc159cc746df None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:36.841625 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93549cf8-3b65-4ba6-af91-bc159cc746df None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:36.845611 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:36.908760 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87078cb5-edb2-48cf-8772-f1574ecd8f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:36.909493 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87078cb5-edb2-48cf-8772-f1574ecd8f87 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:36.909725 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87078cb5-edb2-48cf-8772-f1574ecd8f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:36.910118 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87078cb5-edb2-48cf-8772-f1574ecd8f87 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:36.910901 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87078cb5-edb2-48cf-8772-f1574ecd8f87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:36.913089 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87078cb5-edb2-48cf-8772-f1574ecd8f87 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:36.937530 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 865/1745] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:37:36 2022] POST /identity/v3/users => generated 289 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:37:36.941272 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf6944c-6906-472d-bf79-dee639834abe admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:36.955294 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf6944c-6906-472d-bf79-dee639834abe admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:36.960973 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ebe5cd-0531-42d3-bad2-86f4c0d6bd6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:36.961893 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ebe5cd-0531-42d3-bad2-86f4c0d6bd6d admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:36.962173 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ebe5cd-0531-42d3-bad2-86f4c0d6bd6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:36.962446 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ebe5cd-0531-42d3-bad2-86f4c0d6bd6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:36.963258 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ebe5cd-0531-42d3-bad2-86f4c0d6bd6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:36.965562 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ebe5cd-0531-42d3-bad2-86f4c0d6bd6d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:36.983724 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 881/1746] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:36 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 09:37:36.987528 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87078cb5-edb2-48cf-8772-f1574ecd8f87 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:36.999030 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87078cb5-edb2-48cf-8772-f1574ecd8f87 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:37.005879 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4760ab19-d33f-4904-9ae4-02567f75ed79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:37.007151 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760ab19-d33f-4904-9ae4-02567f75ed79 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:37.007598 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760ab19-d33f-4904-9ae4-02567f75ed79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:37.008024 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760ab19-d33f-4904-9ae4-02567f75ed79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:37.009064 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4760ab19-d33f-4904-9ae4-02567f75ed79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:37.011020 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4760ab19-d33f-4904-9ae4-02567f75ed79 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:37.042659 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 866/1747] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:36 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:37.046112 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ebe5cd-0531-42d3-bad2-86f4c0d6bd6d admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:37.059220 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ebe5cd-0531-42d3-bad2-86f4c0d6bd6d admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:37.072372 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37b99ca-8f93-444a-bb7e-87ddc7b6882d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:37.072967 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b99ca-8f93-444a-bb7e-87ddc7b6882d admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:37.073343 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b99ca-8f93-444a-bb7e-87ddc7b6882d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:37.073678 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b99ca-8f93-444a-bb7e-87ddc7b6882d admin admin] PATH_INFO: `/v3/projects/9b4dc0615d014fd4aecac3e4ffeb95aa/users/c876052410f348aaa0365598f2aa0a33/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:37.103217 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37b99ca-8f93-444a-bb7e-87ddc7b6882d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b4dc0615d014fd4aecac3e4ffeb95aa, user_id=c876052410f348aaa0365598f2aa0a33, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:37.107063 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37b99ca-8f93-444a-bb7e-87ddc7b6882d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:37.166441 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 882/1748] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:37:37 2022] PUT /identity/v3/projects/9b4dc0615d014fd4aecac3e4ffeb95aa/users/c876052410f348aaa0365598f2aa0a33/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:37.168652 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9f4be-9284-4cfe-b401-dc9c94e467db None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:37.169274 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9f4be-9284-4cfe-b401-dc9c94e467db None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:37.169646 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9f4be-9284-4cfe-b401-dc9c94e467db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:37.214865 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-3ea9f4be-9284-4cfe-b401-dc9c94e467db None None] MFA Rules not processed for user `c876052410f348aaa0365598f2aa0a33`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:37.216463 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3ea9f4be-9284-4cfe-b401-dc9c94e467db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:37.224704 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37b99ca-8f93-444a-bb7e-87ddc7b6882d admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:37.226710 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a37b99ca-8f93-444a-bb7e-87ddc7b6882d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:37.348491 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37b99ca-8f93-444a-bb7e-87ddc7b6882d admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:37.354109 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b85a65-2276-48a5-abd5-33720dd6487f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:37.354834 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b85a65-2276-48a5-abd5-33720dd6487f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:37.355135 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b85a65-2276-48a5-abd5-33720dd6487f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:37.355389 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b85a65-2276-48a5-abd5-33720dd6487f admin admin] PATH_INFO: `/v3/users/5f4da0edd6544ad9846e32dc0502e962` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:37.356649 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b85a65-2276-48a5-abd5-33720dd6487f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f4da0edd6544ad9846e32dc0502e962)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:37.359892 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b85a65-2276-48a5-abd5-33720dd6487f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:37.371451 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 867/1749] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:37.372474 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea9f4be-9284-4cfe-b401-dc9c94e467db None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:37.373949 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3ea9f4be-9284-4cfe-b401-dc9c94e467db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:37.390426 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea9f4be-9284-4cfe-b401-dc9c94e467db None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:37.405153 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2006ed2f-86ae-41d8-a51a-1e6c90375e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:37.406502 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2006ed2f-86ae-41d8-a51a-1e6c90375e1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:37.407293 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2006ed2f-86ae-41d8-a51a-1e6c90375e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:37.407755 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2006ed2f-86ae-41d8-a51a-1e6c90375e1c admin admin] PATH_INFO: `/v3/users/23dc03b6ed374e2699c059cc7172b753` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:37.409544 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2006ed2f-86ae-41d8-a51a-1e6c90375e1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=23dc03b6ed374e2699c059cc7172b753)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:37.411553 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2006ed2f-86ae-41d8-a51a-1e6c90375e1c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:37.449391 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-c6b85a65-2276-48a5-abd5-33720dd6487f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f4da0edd6544ad9846e32dc0502e962'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:37.468430 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-c6b85a65-2276-48a5-abd5-33720dd6487f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f4da0edd6544ad9846e32dc0502e962'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:37.486774 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-c6b85a65-2276-48a5-abd5-33720dd6487f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f4da0edd6544ad9846e32dc0502e962'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:37.496625 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-c6b85a65-2276-48a5-abd5-33720dd6487f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f4da0edd6544ad9846e32dc0502e962'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:37.497498 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-c6b85a65-2276-48a5-abd5-33720dd6487f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f4da0edd6544ad9846e32dc0502e962'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:37.499298 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:37.504831 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 883/1750] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:37:37 2022] DELETE /identity/v3/users/5f4da0edd6544ad9846e32dc0502e962 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:37:37.507201 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b85a65-2276-48a5-abd5-33720dd6487f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:37.508918 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c6b85a65-2276-48a5-abd5-33720dd6487f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:37.519821 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-2006ed2f-86ae-41d8-a51a-1e6c90375e1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23dc03b6ed374e2699c059cc7172b753'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:37.530930 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-2006ed2f-86ae-41d8-a51a-1e6c90375e1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23dc03b6ed374e2699c059cc7172b753'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:37.549107 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:37.551936 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-2006ed2f-86ae-41d8-a51a-1e6c90375e1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23dc03b6ed374e2699c059cc7172b753'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:37.570632 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-2006ed2f-86ae-41d8-a51a-1e6c90375e1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23dc03b6ed374e2699c059cc7172b753'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:37.581065 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-2006ed2f-86ae-41d8-a51a-1e6c90375e1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23dc03b6ed374e2699c059cc7172b753'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:37.594917 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 868/1751] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:37:37 2022] DELETE /identity/v3/users/23dc03b6ed374e2699c059cc7172b753 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:37.596415 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110ac86-bff6-4185-9849-7cc729e5e335 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:37.596821 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110ac86-bff6-4185-9849-7cc729e5e335 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:37.597295 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110ac86-bff6-4185-9849-7cc729e5e335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:37.646609 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b85a65-2276-48a5-abd5-33720dd6487f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:37.647888 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a110ac86-bff6-4185-9849-7cc729e5e335 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:37.649461 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a110ac86-bff6-4185-9849-7cc729e5e335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:37.758544 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae171c7-3fed-468e-8904-65100c6aa569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:37.759551 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae171c7-3fed-468e-8904-65100c6aa569 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:37.759999 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae171c7-3fed-468e-8904-65100c6aa569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:37.760423 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae171c7-3fed-468e-8904-65100c6aa569 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:37.761252 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae171c7-3fed-468e-8904-65100c6aa569 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:37.764286 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae171c7-3fed-468e-8904-65100c6aa569 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:37.785800 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 869/1752] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:37.801738 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca259f-745b-45b8-b833-2588f47ac833 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:37.802027 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca259f-745b-45b8-b833-2588f47ac833 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:37.802298 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca259f-745b-45b8-b833-2588f47ac833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:37.815986 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 884/1753] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:37:37 2022] POST /identity/v3/projects => generated 377 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:37:37.817441 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae171c7-3fed-468e-8904-65100c6aa569 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:37.819793 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5ae171c7-3fed-468e-8904-65100c6aa569 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:37.832243 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b1ca259f-745b-45b8-b833-2588f47ac833 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:37.833491 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b1ca259f-745b-45b8-b833-2588f47ac833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:37.906213 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae171c7-3fed-468e-8904-65100c6aa569 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:37.911259 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf428b71-7ff6-4791-81b5-dc671006b524 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:37.911950 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf428b71-7ff6-4791-81b5-dc671006b524 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:37.912224 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf428b71-7ff6-4791-81b5-dc671006b524 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:37.912790 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf428b71-7ff6-4791-81b5-dc671006b524 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:37.955821 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 870/1754] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:37.965440 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca259f-745b-45b8-b833-2588f47ac833 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:37.977760 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b1ca259f-745b-45b8-b833-2588f47ac833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:37.990961 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca259f-745b-45b8-b833-2588f47ac833 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:37.996861 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f5d3d3-50d0-4bda-95d9-9d77b2ecb6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:37.998050 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5d3d3-50d0-4bda-95d9-9d77b2ecb6a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:37.998316 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5d3d3-50d0-4bda-95d9-9d77b2ecb6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:37.998584 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5d3d3-50d0-4bda-95d9-9d77b2ecb6a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:37.999224 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f5d3d3-50d0-4bda-95d9-9d77b2ecb6a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:38.001238 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f5d3d3-50d0-4bda-95d9-9d77b2ecb6a6 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:38.032906 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 871/1755] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:37:37 2022] POST /identity/v3/users => generated 295 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:37:38.037932 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f5d3d3-50d0-4bda-95d9-9d77b2ecb6a6 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:38.043140 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:38.058896 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f5d3d3-50d0-4bda-95d9-9d77b2ecb6a6 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:38.065606 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc09979-5fc6-4ea0-962c-1ee37dfebcd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:38.067204 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc09979-5fc6-4ea0-962c-1ee37dfebcd8 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:38.067649 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc09979-5fc6-4ea0-962c-1ee37dfebcd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:38.068477 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc09979-5fc6-4ea0-962c-1ee37dfebcd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:38.072403 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf428b71-7ff6-4791-81b5-dc671006b524 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:38.074144 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf428b71-7ff6-4791-81b5-dc671006b524 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:38.100720 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 885/1756] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:37:38.102048 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf428b71-7ff6-4791-81b5-dc671006b524 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:38.112324 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc09979-5fc6-4ea0-962c-1ee37dfebcd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:38.114774 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc09979-5fc6-4ea0-962c-1ee37dfebcd8 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:38.124378 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf428b71-7ff6-4791-81b5-dc671006b524 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:38.131586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 872/1757] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:37:38.137783 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a780d6-4957-458a-b40d-9d050784dbe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:38.138471 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a780d6-4957-458a-b40d-9d050784dbe4 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:38.138752 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a780d6-4957-458a-b40d-9d050784dbe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:38.139022 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a780d6-4957-458a-b40d-9d050784dbe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:38.139719 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a780d6-4957-458a-b40d-9d050784dbe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:38.141440 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a780d6-4957-458a-b40d-9d050784dbe4 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:38.158991 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 886/1758] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:38 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:38.162952 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc09979-5fc6-4ea0-962c-1ee37dfebcd8 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:38.174936 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc09979-5fc6-4ea0-962c-1ee37dfebcd8 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:38.180846 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0224ba-146e-4a87-bfd9-e7aae60f52cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:38.181522 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0224ba-146e-4a87-bfd9-e7aae60f52cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:38.181802 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0224ba-146e-4a87-bfd9-e7aae60f52cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:38.182106 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0224ba-146e-4a87-bfd9-e7aae60f52cd admin admin] PATH_INFO: `/v3/projects/eae564db29214c0da8f23285668e435b/users/082d9f6b36474f3386173d70b370e3c0/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:38.218967 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0224ba-146e-4a87-bfd9-e7aae60f52cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=eae564db29214c0da8f23285668e435b, user_id=082d9f6b36474f3386173d70b370e3c0, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:38.223048 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0224ba-146e-4a87-bfd9-e7aae60f52cd admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:38.249563 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 873/1759] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:37:38 2022] PUT /identity/v3/projects/eae564db29214c0da8f23285668e435b/users/082d9f6b36474f3386173d70b370e3c0/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:37:38.253132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a780d6-4957-458a-b40d-9d050784dbe4 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:38.266650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a780d6-4957-458a-b40d-9d050784dbe4 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:38.348697 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b6e0c4-6e2e-4779-a710-d887d75084b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:38.349320 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6e0c4-6e2e-4779-a710-d887d75084b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:38.349598 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6e0c4-6e2e-4779-a710-d887d75084b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:38.349869 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6e0c4-6e2e-4779-a710-d887d75084b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:38.350505 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b6e0c4-6e2e-4779-a710-d887d75084b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:38.352203 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b6e0c4-6e2e-4779-a710-d887d75084b8 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:38.364872 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 887/1760] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:38 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:38.369194 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0224ba-146e-4a87-bfd9-e7aae60f52cd admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:38.380088 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0224ba-146e-4a87-bfd9-e7aae60f52cd admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:38.385300 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a1ee8f-0e13-4c27-9d30-d5f83c0f0099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:38.386032 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1ee8f-0e13-4c27-9d30-d5f83c0f0099 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:38.386883 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1ee8f-0e13-4c27-9d30-d5f83c0f0099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:38.386883 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1ee8f-0e13-4c27-9d30-d5f83c0f0099 admin admin] PATH_INFO: `/v3/projects/eae564db29214c0da8f23285668e435b/users/082d9f6b36474f3386173d70b370e3c0/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:38.389035 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a1ee8f-0e13-4c27-9d30-d5f83c0f0099 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eae564db29214c0da8f23285668e435b, user_id=082d9f6b36474f3386173d70b370e3c0, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:38.392222 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a1ee8f-0e13-4c27-9d30-d5f83c0f0099 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:38.405745 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 874/1761] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:37:38 2022] PUT /identity/v3/projects/eae564db29214c0da8f23285668e435b/users/082d9f6b36474f3386173d70b370e3c0/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:38.417409 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4eab27-23ab-47ad-9a20-e760fa217f66 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:38.417823 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4eab27-23ab-47ad-9a20-e760fa217f66 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:38.418191 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4eab27-23ab-47ad-9a20-e760fa217f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:38.454843 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-ee4eab27-23ab-47ad-9a20-e760fa217f66 None None] MFA Rules not processed for user `082d9f6b36474f3386173d70b370e3c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:38.457849 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ee4eab27-23ab-47ad-9a20-e760fa217f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:38.556746 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 888/1762] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:38 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:38.568565 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a1ee8f-0e13-4c27-9d30-d5f83c0f0099 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:38.571710 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-18a1ee8f-0e13-4c27-9d30-d5f83c0f0099 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:38.610844 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:DELETE=2,keystone:SELECT=92,keystone:INSERT=6 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:38.655271 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4eab27-23ab-47ad-9a20-e760fa217f66 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:38.657825 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ee4eab27-23ab-47ad-9a20-e760fa217f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:38.681351 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a1ee8f-0e13-4c27-9d30-d5f83c0f0099 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:38.687009 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799389a1-c210-490d-9c08-2527a957623d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:38.688049 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799389a1-c210-490d-9c08-2527a957623d service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:38.688335 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799389a1-c210-490d-9c08-2527a957623d service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:38.688673 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799389a1-c210-490d-9c08-2527a957623d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:38.691568 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-799389a1-c210-490d-9c08-2527a957623d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:38.749615 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:38.763873 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799389a1-c210-490d-9c08-2527a957623d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:38.764275 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4eab27-23ab-47ad-9a20-e760fa217f66 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:38.765337 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799389a1-c210-490d-9c08-2527a957623d service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:38.769095 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad86259-37ff-4e1a-92e1-0466ccf1dc62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:38.769942 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad86259-37ff-4e1a-92e1-0466ccf1dc62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:38.770226 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad86259-37ff-4e1a-92e1-0466ccf1dc62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:38.770499 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad86259-37ff-4e1a-92e1-0466ccf1dc62 admin admin] PATH_INFO: `/v3/projects/1cf82af3dca14828a82f164f538668a5` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:38.771603 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad86259-37ff-4e1a-92e1-0466ccf1dc62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cf82af3dca14828a82f164f538668a5)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:38.773523 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad86259-37ff-4e1a-92e1-0466ccf1dc62 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:38.774336 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 875/1763] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:37:38.775116 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799389a1-c210-490d-9c08-2527a957623d service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:38.776389 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-799389a1-c210-490d-9c08-2527a957623d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:38.790925 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799389a1-c210-490d-9c08-2527a957623d service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:38.796778 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6902cba8-c804-4485-83ab-9d61fdda9682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:38.797489 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902cba8-c804-4485-83ab-9d61fdda9682 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:38.797844 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902cba8-c804-4485-83ab-9d61fdda9682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:38.798182 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902cba8-c804-4485-83ab-9d61fdda9682 admin admin] PATH_INFO: `/v3/projects/ff934d8402614cca93a0c8a5e36a177f` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:38.799492 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6902cba8-c804-4485-83ab-9d61fdda9682 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff934d8402614cca93a0c8a5e36a177f)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:38.801770 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6902cba8-c804-4485-83ab-9d61fdda9682 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:38.859076 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-bad86259-37ff-4e1a-92e1-0466ccf1dc62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cf82af3dca14828a82f164f538668a5', 'request_id': 'req-bad86259-37ff-4e1a-92e1-0466ccf1dc62'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:38.867652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-bad86259-37ff-4e1a-92e1-0466ccf1dc62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cf82af3dca14828a82f164f538668a5', 'request_id': 'req-bad86259-37ff-4e1a-92e1-0466ccf1dc62'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:38.875092 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6902cba8-c804-4485-83ab-9d61fdda9682 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff934d8402614cca93a0c8a5e36a177f', 'request_id': 'req-6902cba8-c804-4485-83ab-9d61fdda9682'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:38.877715 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-bad86259-37ff-4e1a-92e1-0466ccf1dc62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cf82af3dca14828a82f164f538668a5', 'request_id': 'req-bad86259-37ff-4e1a-92e1-0466ccf1dc62'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:38.885199 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6902cba8-c804-4485-83ab-9d61fdda9682 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff934d8402614cca93a0c8a5e36a177f', 'request_id': 'req-6902cba8-c804-4485-83ab-9d61fdda9682'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:38.892442 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6902cba8-c804-4485-83ab-9d61fdda9682 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff934d8402614cca93a0c8a5e36a177f', 'request_id': 'req-6902cba8-c804-4485-83ab-9d61fdda9682'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:38.896735 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-bad86259-37ff-4e1a-92e1-0466ccf1dc62 admin admin] The token cache is being invalidate because project 1cf82af3dca14828a82f164f538668a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:37:38.897369 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-bad86259-37ff-4e1a-92e1-0466ccf1dc62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:38.898332 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-bad86259-37ff-4e1a-92e1-0466ccf1dc62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:38.900392 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 889/1764] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:37:38 2022] DELETE /identity/v3/projects/1cf82af3dca14828a82f164f538668a5 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:37:38.908791 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad86259-37ff-4e1a-92e1-0466ccf1dc62 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:38.910283 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-bad86259-37ff-4e1a-92e1-0466ccf1dc62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:38.910639 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6902cba8-c804-4485-83ab-9d61fdda9682 admin admin] The token cache is being invalidate because project ff934d8402614cca93a0c8a5e36a177f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:37:38.911191 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6902cba8-c804-4485-83ab-9d61fdda9682 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:38.912034 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6902cba8-c804-4485-83ab-9d61fdda9682 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:38.913764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 876/1765] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:37:38 2022] DELETE /identity/v3/projects/ff934d8402614cca93a0c8a5e36a177f => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:38.918611 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6be57-7c38-45b2-a46b-db6fbaa8cb65 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:38.919971 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6be57-7c38-45b2-a46b-db6fbaa8cb65 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:38.920894 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6be57-7c38-45b2-a46b-db6fbaa8cb65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:38.968632 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-5cb6be57-7c38-45b2-a46b-db6fbaa8cb65 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:38.973302 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5cb6be57-7c38-45b2-a46b-db6fbaa8cb65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:39.001282 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad86259-37ff-4e1a-92e1-0466ccf1dc62 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:39.054394 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:39.066412 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 877/1766] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:39.073969 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef2c6d-716e-41eb-b679-ee125fefec3b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:39.074347 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef2c6d-716e-41eb-b679-ee125fefec3b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:39.074867 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef2c6d-716e-41eb-b679-ee125fefec3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:39.076192 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e72778-6e60-40e3-9bd8-5260b43a65f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:39.077002 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e72778-6e60-40e3-9bd8-5260b43a65f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:39.077282 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e72778-6e60-40e3-9bd8-5260b43a65f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:39.077590 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e72778-6e60-40e3-9bd8-5260b43a65f8 admin admin] PATH_INFO: `/v3/users/e5083e3ac6854a41bf1fc5b98fa8613d` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:39.078852 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e72778-6e60-40e3-9bd8-5260b43a65f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5083e3ac6854a41bf1fc5b98fa8613d)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:39.081207 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e72778-6e60-40e3-9bd8-5260b43a65f8 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:39.108369 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-53ef2c6d-716e-41eb-b679-ee125fefec3b None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:39.110249 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-53ef2c6d-716e-41eb-b679-ee125fefec3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:39.135135 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 878/1767] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:39.139387 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ef2c6d-716e-41eb-b679-ee125fefec3b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:39.141673 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:39.150237 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ef2c6d-716e-41eb-b679-ee125fefec3b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:39.164540 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb879a6a-bd76-486a-93fc-1719588dd668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:39.165353 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb879a6a-bd76-486a-93fc-1719588dd668 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:39.165634 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb879a6a-bd76-486a-93fc-1719588dd668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:39.165905 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb879a6a-bd76-486a-93fc-1719588dd668 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:39.166781 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb879a6a-bd76-486a-93fc-1719588dd668 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:37:39.167087 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb879a6a-bd76-486a-93fc-1719588dd668 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:39.168762 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb879a6a-bd76-486a-93fc-1719588dd668 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:39.171970 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-38e72778-6e60-40e3-9bd8-5260b43a65f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5083e3ac6854a41bf1fc5b98fa8613d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:39.180032 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-38e72778-6e60-40e3-9bd8-5260b43a65f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5083e3ac6854a41bf1fc5b98fa8613d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:39.182970 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 879/1768] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:37:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:37:39.187381 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb879a6a-bd76-486a-93fc-1719588dd668 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:39.200417 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-38e72778-6e60-40e3-9bd8-5260b43a65f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5083e3ac6854a41bf1fc5b98fa8613d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:39.207027 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb879a6a-bd76-486a-93fc-1719588dd668 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:39.211629 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-38e72778-6e60-40e3-9bd8-5260b43a65f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5083e3ac6854a41bf1fc5b98fa8613d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:39.212353 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-38e72778-6e60-40e3-9bd8-5260b43a65f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5083e3ac6854a41bf1fc5b98fa8613d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:39.212737 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8564e85-5546-4c80-a8e1-e3415c5b10c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:39.213314 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8564e85-5546-4c80-a8e1-e3415c5b10c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:39.213627 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8564e85-5546-4c80-a8e1-e3415c5b10c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:39.213931 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8564e85-5546-4c80-a8e1-e3415c5b10c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:39.214549 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8564e85-5546-4c80-a8e1-e3415c5b10c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:39.217098 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8564e85-5546-4c80-a8e1-e3415c5b10c4 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:39.222331 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 890/1769] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:37:38 2022] DELETE /identity/v3/users/e5083e3ac6854a41bf1fc5b98fa8613d => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:39.255148 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 880/1770] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:37:39 2022] POST /identity/v3/projects => generated 385 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:37:39.256259 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e72778-6e60-40e3-9bd8-5260b43a65f8 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:39.257831 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-38e72778-6e60-40e3-9bd8-5260b43a65f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:39.348723 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e72778-6e60-40e3-9bd8-5260b43a65f8 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:39.411124 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bad8da-20ae-4017-97a1-d20898ec207a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:39.411779 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bad8da-20ae-4017-97a1-d20898ec207a admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:39.412091 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bad8da-20ae-4017-97a1-d20898ec207a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:39.412368 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bad8da-20ae-4017-97a1-d20898ec207a admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:39.413060 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bad8da-20ae-4017-97a1-d20898ec207a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:39.415014 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bad8da-20ae-4017-97a1-d20898ec207a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:39.438751 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 891/1771] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:37:39 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:37:39.442798 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8564e85-5546-4c80-a8e1-e3415c5b10c4 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:39.454284 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8564e85-5546-4c80-a8e1-e3415c5b10c4 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:39.460504 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be6a0b1-ba94-49e7-a18f-18d1daa2577c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:39.461171 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be6a0b1-ba94-49e7-a18f-18d1daa2577c admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:39.461451 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be6a0b1-ba94-49e7-a18f-18d1daa2577c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:39.461721 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be6a0b1-ba94-49e7-a18f-18d1daa2577c admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:39.462362 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be6a0b1-ba94-49e7-a18f-18d1daa2577c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:39.463971 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be6a0b1-ba94-49e7-a18f-18d1daa2577c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:39.474828 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 881/1772] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:39.478782 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bad8da-20ae-4017-97a1-d20898ec207a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:39.489472 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bad8da-20ae-4017-97a1-d20898ec207a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:39.494088 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7402a1-0d10-4cea-96c4-97d24a826155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:39.494694 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7402a1-0d10-4cea-96c4-97d24a826155 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:39.494975 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7402a1-0d10-4cea-96c4-97d24a826155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:39.495244 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7402a1-0d10-4cea-96c4-97d24a826155 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:39.495874 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7402a1-0d10-4cea-96c4-97d24a826155 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:39.497521 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7402a1-0d10-4cea-96c4-97d24a826155 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:39.507896 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 892/1773] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:39.512325 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be6a0b1-ba94-49e7-a18f-18d1daa2577c admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:39.524890 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be6a0b1-ba94-49e7-a18f-18d1daa2577c admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:39.530054 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e58ee6c-ca4d-47bb-ad9b-866281792a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:39.530629 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7402a1-0d10-4cea-96c4-97d24a826155 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:39.530752 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58ee6c-ca4d-47bb-ad9b-866281792a4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:39.530972 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58ee6c-ca4d-47bb-ad9b-866281792a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:39.531303 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58ee6c-ca4d-47bb-ad9b-866281792a4b admin admin] PATH_INFO: `/v3/projects/7529b42d0cd24f23a9023a9728bbf1c6/users/37f0f7cc992e447a9ea08e711e226e4f/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:39.532742 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8e7402a1-0d10-4cea-96c4-97d24a826155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:39.553406 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7402a1-0d10-4cea-96c4-97d24a826155 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:39.558874 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ed5618-28aa-4bd4-b1b0-d561108daf79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:39.559514 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed5618-28aa-4bd4-b1b0-d561108daf79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:39.559802 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed5618-28aa-4bd4-b1b0-d561108daf79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:39.560066 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed5618-28aa-4bd4-b1b0-d561108daf79 admin admin] PATH_INFO: `/v3/projects/4c5c9ffc26c74c0d92495ef7653318ce` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:39.561179 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ed5618-28aa-4bd4-b1b0-d561108daf79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c5c9ffc26c74c0d92495ef7653318ce)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:39.562646 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e58ee6c-ca4d-47bb-ad9b-866281792a4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7529b42d0cd24f23a9023a9728bbf1c6, user_id=37f0f7cc992e447a9ea08e711e226e4f, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:39.563087 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ed5618-28aa-4bd4-b1b0-d561108daf79 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:39.565955 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e58ee6c-ca4d-47bb-ad9b-866281792a4b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:39.581633 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 882/1774] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:37:39 2022] PUT /identity/v3/projects/7529b42d0cd24f23a9023a9728bbf1c6/users/37f0f7cc992e447a9ea08e711e226e4f/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:39.592276 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62cf590-be3c-4c73-bbe6-09b1205a5b3a None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:39.592582 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62cf590-be3c-4c73-bbe6-09b1205a5b3a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:39.592855 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62cf590-be3c-4c73-bbe6-09b1205a5b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:39.600440 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:39.620030 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-68ed5618-28aa-4bd4-b1b0-d561108daf79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c5c9ffc26c74c0d92495ef7653318ce', 'request_id': 'req-68ed5618-28aa-4bd4-b1b0-d561108daf79'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:39.622112 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a62cf590-be3c-4c73-bbe6-09b1205a5b3a None None] MFA Rules not processed for user `37f0f7cc992e447a9ea08e711e226e4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:39.623227 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a62cf590-be3c-4c73-bbe6-09b1205a5b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:39.626176 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-68ed5618-28aa-4bd4-b1b0-d561108daf79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c5c9ffc26c74c0d92495ef7653318ce', 'request_id': 'req-68ed5618-28aa-4bd4-b1b0-d561108daf79'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:39.634530 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-68ed5618-28aa-4bd4-b1b0-d561108daf79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c5c9ffc26c74c0d92495ef7653318ce', 'request_id': 'req-68ed5618-28aa-4bd4-b1b0-d561108daf79'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:39.647465 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-68ed5618-28aa-4bd4-b1b0-d561108daf79 admin admin] The token cache is being invalidate because project 4c5c9ffc26c74c0d92495ef7653318ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:37:39.647754 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-68ed5618-28aa-4bd4-b1b0-d561108daf79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:39.648436 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-68ed5618-28aa-4bd4-b1b0-d561108daf79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:39.650013 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 893/1775] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:37:39 2022] DELETE /identity/v3/projects/4c5c9ffc26c74c0d92495ef7653318ce => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:39.656639 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09090f38-8b31-45a9-9eae-96cb3aef42c2 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:39.657014 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09090f38-8b31-45a9-9eae-96cb3aef42c2 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:39.657388 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09090f38-8b31-45a9-9eae-96cb3aef42c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:39.682986 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-09090f38-8b31-45a9-9eae-96cb3aef42c2 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:39.684191 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-09090f38-8b31-45a9-9eae-96cb3aef42c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:39.700828 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 883/1776] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:37:39.702153 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62cf590-be3c-4c73-bbe6-09b1205a5b3a None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:39.703484 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a62cf590-be3c-4c73-bbe6-09b1205a5b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:39.756608 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 894/1777] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:37:39.758308 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a623c1-5b95-429c-8abf-7202b59c2d99 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:39.758588 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a623c1-5b95-429c-8abf-7202b59c2d99 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:39.759057 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a623c1-5b95-429c-8abf-7202b59c2d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:39.781267 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-c2a623c1-5b95-429c-8abf-7202b59c2d99 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:39.782565 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c2a623c1-5b95-429c-8abf-7202b59c2d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:39.787122 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62cf590-be3c-4c73-bbe6-09b1205a5b3a None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:39.792391 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ef1c98-9881-4122-972e-1c885d225d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:39.796010 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 895/1778] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:39.796324 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ef1c98-9881-4122-972e-1c885d225d53 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:39.796324 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ef1c98-9881-4122-972e-1c885d225d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:39.796324 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ef1c98-9881-4122-972e-1c885d225d53 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:39.796324 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ef1c98-9881-4122-972e-1c885d225d53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:39.796619 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ef1c98-9881-4122-972e-1c885d225d53 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:39.796977 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a623c1-5b95-429c-8abf-7202b59c2d99 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:39.806841 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a623c1-5b95-429c-8abf-7202b59c2d99 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:39.812256 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03248eb2-ea66-4141-bf42-b7c7cafc62d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:39.812884 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03248eb2-ea66-4141-bf42-b7c7cafc62d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:39.813041 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:39.813147 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03248eb2-ea66-4141-bf42-b7c7cafc62d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:39.813409 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03248eb2-ea66-4141-bf42-b7c7cafc62d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:39.814045 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03248eb2-ea66-4141-bf42-b7c7cafc62d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:37:39.814453 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03248eb2-ea66-4141-bf42-b7c7cafc62d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:39.816393 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03248eb2-ea66-4141-bf42-b7c7cafc62d7 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:39.827924 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 884/1779] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:37:39 2022] POST /identity/v3/projects => generated 385 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:37:39.832904 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 896/1780] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:37:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:37:39.833038 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ef1c98-9881-4122-972e-1c885d225d53 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:39.833899 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03248eb2-ea66-4141-bf42-b7c7cafc62d7 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:39.845881 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03248eb2-ea66-4141-bf42-b7c7cafc62d7 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:39.903951 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ef1c98-9881-4122-972e-1c885d225d53 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:39.904168 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb55145-c28e-489b-8265-ed0bb6f2d011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:39.904729 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb55145-c28e-489b-8265-ed0bb6f2d011 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:39.905011 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb55145-c28e-489b-8265-ed0bb6f2d011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:39.905289 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb55145-c28e-489b-8265-ed0bb6f2d011 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:39.905918 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb55145-c28e-489b-8265-ed0bb6f2d011 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:39.907809 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb55145-c28e-489b-8265-ed0bb6f2d011 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:39.908814 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa3205c-69d8-4254-a7f7-313f858bfe90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:39.909448 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa3205c-69d8-4254-a7f7-313f858bfe90 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:39.909721 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa3205c-69d8-4254-a7f7-313f858bfe90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:39.909985 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa3205c-69d8-4254-a7f7-313f858bfe90 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:39.910609 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa3205c-69d8-4254-a7f7-313f858bfe90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:39.912579 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa3205c-69d8-4254-a7f7-313f858bfe90 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:39.936251 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 885/1781] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:37:39 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:37:39.939837 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa3205c-69d8-4254-a7f7-313f858bfe90 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:39.944644 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 897/1782] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:37:39 2022] POST /identity/v3/projects => generated 377 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:37:39.948505 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb55145-c28e-489b-8265-ed0bb6f2d011 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:39.949770 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa3205c-69d8-4254-a7f7-313f858bfe90 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:39.972945 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:40.015535 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb55145-c28e-489b-8265-ed0bb6f2d011 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:40.018075 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818632cd-ccce-4c8f-8682-51bb9b8f4a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:40.018800 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818632cd-ccce-4c8f-8682-51bb9b8f4a84 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:40.019081 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818632cd-ccce-4c8f-8682-51bb9b8f4a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:40.019347 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818632cd-ccce-4c8f-8682-51bb9b8f4a84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:40.019979 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818632cd-ccce-4c8f-8682-51bb9b8f4a84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:40.020197 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21c19da-8477-433b-9216-40d94c0af69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:40.020832 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21c19da-8477-433b-9216-40d94c0af69b admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:40.021111 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21c19da-8477-433b-9216-40d94c0af69b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:40.021398 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21c19da-8477-433b-9216-40d94c0af69b admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:40.021602 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818632cd-ccce-4c8f-8682-51bb9b8f4a84 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:40.022003 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21c19da-8477-433b-9216-40d94c0af69b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:40.025367 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21c19da-8477-433b-9216-40d94c0af69b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:40.032102 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 886/1783] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:39 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 09:37:40.036366 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818632cd-ccce-4c8f-8682-51bb9b8f4a84 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:40.048305 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818632cd-ccce-4c8f-8682-51bb9b8f4a84 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:40.051518 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 898/1784] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:37:39 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:37:40.055841 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21c19da-8477-433b-9216-40d94c0af69b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:40.057303 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a83613-a9a4-4d84-b164-cee72763e081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:40.058193 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a83613-a9a4-4d84-b164-cee72763e081 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:40.058617 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a83613-a9a4-4d84-b164-cee72763e081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:40.059056 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a83613-a9a4-4d84-b164-cee72763e081 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:40.060389 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a83613-a9a4-4d84-b164-cee72763e081 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:40.062881 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a83613-a9a4-4d84-b164-cee72763e081 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:40.066102 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21c19da-8477-433b-9216-40d94c0af69b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:40.071002 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a076c8a0-3df0-4529-941f-a90b83cc8381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:40.071606 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076c8a0-3df0-4529-941f-a90b83cc8381 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:40.071884 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076c8a0-3df0-4529-941f-a90b83cc8381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:40.072159 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076c8a0-3df0-4529-941f-a90b83cc8381 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:40.072824 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a076c8a0-3df0-4529-941f-a90b83cc8381 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:40.074401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a076c8a0-3df0-4529-941f-a90b83cc8381 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:40.084561 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 887/1785] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:40 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:40.088101 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a83613-a9a4-4d84-b164-cee72763e081 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:40.092827 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 899/1786] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:40.096642 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a076c8a0-3df0-4529-941f-a90b83cc8381 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:40.099636 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a83613-a9a4-4d84-b164-cee72763e081 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:40.105406 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075f9a67-a8a6-4d8f-86be-6c7b017bf52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:40.106088 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f9a67-a8a6-4d8f-86be-6c7b017bf52c admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:40.106410 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f9a67-a8a6-4d8f-86be-6c7b017bf52c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:40.106547 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a076c8a0-3df0-4529-941f-a90b83cc8381 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:40.106701 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f9a67-a8a6-4d8f-86be-6c7b017bf52c admin admin] PATH_INFO: `/v3/projects/fb8fcb81ff684d83a73b114978b1d199/users/411e5dd885344de7b7a88b2a9662433f/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:40.113287 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab18d1e-9f8d-4de3-8068-dcdb18f7ec0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:40.113868 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab18d1e-9f8d-4de3-8068-dcdb18f7ec0e admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:40.114149 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab18d1e-9f8d-4de3-8068-dcdb18f7ec0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:40.114451 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab18d1e-9f8d-4de3-8068-dcdb18f7ec0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:40.115091 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab18d1e-9f8d-4de3-8068-dcdb18f7ec0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:40.116734 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab18d1e-9f8d-4de3-8068-dcdb18f7ec0e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:40.124028 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075f9a67-a8a6-4d8f-86be-6c7b017bf52c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb8fcb81ff684d83a73b114978b1d199, user_id=411e5dd885344de7b7a88b2a9662433f, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:40.127447 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075f9a67-a8a6-4d8f-86be-6c7b017bf52c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:40.129376 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 900/1787] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 09:37:40.133545 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab18d1e-9f8d-4de3-8068-dcdb18f7ec0e admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:40.143150 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 888/1788] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:37:40 2022] PUT /identity/v3/projects/fb8fcb81ff684d83a73b114978b1d199/users/411e5dd885344de7b7a88b2a9662433f/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:37:40.154004 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab18d1e-9f8d-4de3-8068-dcdb18f7ec0e admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:40.154191 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9f513e-2df7-4a53-8102-ee444f15245a None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:40.154305 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9f513e-2df7-4a53-8102-ee444f15245a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:40.154544 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9f513e-2df7-4a53-8102-ee444f15245a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:40.201193 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-3d9f513e-2df7-4a53-8102-ee444f15245a None None] MFA Rules not processed for user `411e5dd885344de7b7a88b2a9662433f`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:40.202795 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3d9f513e-2df7-4a53-8102-ee444f15245a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:40.226499 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26a30e7-b675-48e8-979c-b7e33732a7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:40.227614 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a30e7-b675-48e8-979c-b7e33732a7cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:40.227899 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a30e7-b675-48e8-979c-b7e33732a7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:40.228173 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a30e7-b675-48e8-979c-b7e33732a7cb admin admin] PATH_INFO: `/v3/projects/c72f89f60e2245e2aa5c287ca187cfa2/users/2fd9e974ca4b47fcad9004a0f474094b/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:40.244252 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:40.248706 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26a30e7-b675-48e8-979c-b7e33732a7cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c72f89f60e2245e2aa5c287ca187cfa2, user_id=2fd9e974ca4b47fcad9004a0f474094b, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:40.251644 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26a30e7-b675-48e8-979c-b7e33732a7cb admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:40.270865 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 901/1789] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:37:40 2022] PUT /identity/v3/projects/c72f89f60e2245e2aa5c287ca187cfa2/users/2fd9e974ca4b47fcad9004a0f474094b/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:37:40.285066 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1652ac1a-27f3-4c74-9116-cfdd775ed680 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:40.285066 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1652ac1a-27f3-4c74-9116-cfdd775ed680 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:40.285308 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1652ac1a-27f3-4c74-9116-cfdd775ed680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:40.291251 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 889/1790] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:37:40.298593 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9f513e-2df7-4a53-8102-ee444f15245a None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:40.310807 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9f513e-2df7-4a53-8102-ee444f15245a None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:40.321918 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-1652ac1a-27f3-4c74-9116-cfdd775ed680 None None] MFA Rules not processed for user `2fd9e974ca4b47fcad9004a0f474094b`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:40.323168 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1652ac1a-27f3-4c74-9116-cfdd775ed680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:40.400000 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f682b-b328-4440-bad8-c80e5ccaf378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:40.400658 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f682b-b328-4440-bad8-c80e5ccaf378 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:40.400939 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f682b-b328-4440-bad8-c80e5ccaf378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:40.401205 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f682b-b328-4440-bad8-c80e5ccaf378 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:40.401839 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145f682b-b328-4440-bad8-c80e5ccaf378 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:40.403811 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145f682b-b328-4440-bad8-c80e5ccaf378 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:40.423887 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 902/1791] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:40.434020 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1652ac1a-27f3-4c74-9116-cfdd775ed680 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:40.436152 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1652ac1a-27f3-4c74-9116-cfdd775ed680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:40.443969 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 890/1792] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:37:40 2022] POST /identity/v3/projects => generated 383 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:37:40.445614 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:40.446510 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f682b-b328-4440-bad8-c80e5ccaf378 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:40.502769 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1652ac1a-27f3-4c74-9116-cfdd775ed680 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:40.515136 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f682b-b328-4440-bad8-c80e5ccaf378 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:40.520224 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1619c843-50f2-4dd2-862c-c436822055c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:40.520921 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1619c843-50f2-4dd2-862c-c436822055c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:40.521143 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1619c843-50f2-4dd2-862c-c436822055c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:40.521414 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1619c843-50f2-4dd2-862c-c436822055c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:40.522040 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1619c843-50f2-4dd2-862c-c436822055c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:40.523961 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1619c843-50f2-4dd2-862c-c436822055c7 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:40.551719 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 891/1793] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:37:40 2022] POST /identity/v3/users => generated 298 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:37:40.551719 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1619c843-50f2-4dd2-862c-c436822055c7 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:40.551719 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1619c843-50f2-4dd2-862c-c436822055c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:40.565236 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb200f-43c8-4d8e-b72d-57998b9ee301 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:40.565754 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb200f-43c8-4d8e-b72d-57998b9ee301 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:40.566042 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb200f-43c8-4d8e-b72d-57998b9ee301 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:40.566410 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb200f-43c8-4d8e-b72d-57998b9ee301 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:40.611965 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1619c843-50f2-4dd2-862c-c436822055c7 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:40.615745 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d55ea9-61c6-488c-b497-0c35b46e2dc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:40.616646 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d55ea9-61c6-488c-b497-0c35b46e2dc2 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:40.616930 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d55ea9-61c6-488c-b497-0c35b46e2dc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:40.617404 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d55ea9-61c6-488c-b497-0c35b46e2dc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:40.619695 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-39d55ea9-61c6-488c-b497-0c35b46e2dc2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:40.624725 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:40.627138 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcb200f-43c8-4d8e-b72d-57998b9ee301 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:40.628706 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcb200f-43c8-4d8e-b72d-57998b9ee301 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:40.638243 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 903/1794] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:40 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 09:37:40.639196 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb200f-43c8-4d8e-b72d-57998b9ee301 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:40.653013 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb200f-43c8-4d8e-b72d-57998b9ee301 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:40.660165 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8c3461-649d-493f-8317-c4d71d914e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:40.660849 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c3461-649d-493f-8317-c4d71d914e21 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:40.661131 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c3461-649d-493f-8317-c4d71d914e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:40.661405 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c3461-649d-493f-8317-c4d71d914e21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:40.662051 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8c3461-649d-493f-8317-c4d71d914e21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:40.663639 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8c3461-649d-493f-8317-c4d71d914e21 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:40.675563 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 904/1795] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:40.679612 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8c3461-649d-493f-8317-c4d71d914e21 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:40.690736 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8c3461-649d-493f-8317-c4d71d914e21 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:40.693302 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d55ea9-61c6-488c-b497-0c35b46e2dc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:40.694761 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d55ea9-61c6-488c-b497-0c35b46e2dc2 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:40.695521 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff84d280-e9c3-422e-918e-be8c18098461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:40.696171 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84d280-e9c3-422e-918e-be8c18098461 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:40.696627 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84d280-e9c3-422e-918e-be8c18098461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:40.696900 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84d280-e9c3-422e-918e-be8c18098461 admin admin] PATH_INFO: `/v3/projects/38d5a8cd58ec459283642d104dc2aa5a/users/b4d1f626514049f7839fe241d466d98f/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:40.709153 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 892/1796] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:40 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:37:40.717974 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff84d280-e9c3-422e-918e-be8c18098461 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38d5a8cd58ec459283642d104dc2aa5a, user_id=b4d1f626514049f7839fe241d466d98f, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:40.720510 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff84d280-e9c3-422e-918e-be8c18098461 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:40.741759 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 905/1797] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:37:40 2022] PUT /identity/v3/projects/38d5a8cd58ec459283642d104dc2aa5a/users/b4d1f626514049f7839fe241d466d98f/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:40.743236 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d55ea9-61c6-488c-b497-0c35b46e2dc2 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:40.756970 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d55ea9-61c6-488c-b497-0c35b46e2dc2 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:40.832833 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa546a61-2296-4547-9e7c-56acf3ee8881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:40.833729 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa546a61-2296-4547-9e7c-56acf3ee8881 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:40.833866 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa546a61-2296-4547-9e7c-56acf3ee8881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:40.834126 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa546a61-2296-4547-9e7c-56acf3ee8881 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:40.835030 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa546a61-2296-4547-9e7c-56acf3ee8881 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:40.837274 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa546a61-2296-4547-9e7c-56acf3ee8881 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:40.850451 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 893/1798] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:40 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:40.854987 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff84d280-e9c3-422e-918e-be8c18098461 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:40.866250 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff84d280-e9c3-422e-918e-be8c18098461 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:40.870948 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d9aa79-a70c-47cd-9904-148476c6c241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:40.871602 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d9aa79-a70c-47cd-9904-148476c6c241 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:40.871882 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d9aa79-a70c-47cd-9904-148476c6c241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:40.872180 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d9aa79-a70c-47cd-9904-148476c6c241 admin admin] PATH_INFO: `/v3/projects/38d5a8cd58ec459283642d104dc2aa5a/users/b4d1f626514049f7839fe241d466d98f/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:40.874309 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d9aa79-a70c-47cd-9904-148476c6c241 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38d5a8cd58ec459283642d104dc2aa5a, user_id=b4d1f626514049f7839fe241d466d98f, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:40.877033 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d9aa79-a70c-47cd-9904-148476c6c241 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:40.891110 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 906/1799] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:37:40 2022] PUT /identity/v3/projects/38d5a8cd58ec459283642d104dc2aa5a/users/b4d1f626514049f7839fe241d466d98f/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:40.901633 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bac6b7-4115-4b57-9b1c-145583cbd01d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:40.901986 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bac6b7-4115-4b57-9b1c-145583cbd01d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:40.902328 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bac6b7-4115-4b57-9b1c-145583cbd01d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:40.939733 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-e6bac6b7-4115-4b57-9b1c-145583cbd01d None None] MFA Rules not processed for user `b4d1f626514049f7839fe241d466d98f`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:40.941913 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e6bac6b7-4115-4b57-9b1c-145583cbd01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:41.065446 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 894/1800] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:40 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:41.077386 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d9aa79-a70c-47cd-9904-148476c6c241 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:41.104714 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:41.134128 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d9aa79-a70c-47cd-9904-148476c6c241 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:41.138246 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ee08be-58e3-4736-b00a-68c2130704fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:41.138845 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee08be-58e3-4736-b00a-68c2130704fc service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:41.139113 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee08be-58e3-4736-b00a-68c2130704fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:41.139373 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee08be-58e3-4736-b00a-68c2130704fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:41.199703 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ee08be-58e3-4736-b00a-68c2130704fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:41.201813 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ee08be-58e3-4736-b00a-68c2130704fc service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:41.211562 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 907/1801] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:37:41.489573 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bac6b7-4115-4b57-9b1c-145583cbd01d None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:41.502457 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bac6b7-4115-4b57-9b1c-145583cbd01d None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:41.506171 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a528511-4998-4281-bf0e-284237c9f1cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:41.506857 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a528511-4998-4281-bf0e-284237c9f1cf service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:41.507191 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a528511-4998-4281-bf0e-284237c9f1cf service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:41.507533 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a528511-4998-4281-bf0e-284237c9f1cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:41.509841 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1a528511-4998-4281-bf0e-284237c9f1cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:41.540274 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:41.589044 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a528511-4998-4281-bf0e-284237c9f1cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:41.590420 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a528511-4998-4281-bf0e-284237c9f1cf service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:41.600746 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 895/1802] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:41 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:37:42.810236 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ee08be-58e3-4736-b00a-68c2130704fc service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:42.819533 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ee08be-58e3-4736-b00a-68c2130704fc service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:42.823677 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6251ae-9e00-458c-988e-acf3a062b5d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:42.824292 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6251ae-9e00-458c-988e-acf3a062b5d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:42.824603 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6251ae-9e00-458c-988e-acf3a062b5d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:42.824878 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6251ae-9e00-458c-988e-acf3a062b5d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:42.884193 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6251ae-9e00-458c-988e-acf3a062b5d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:42.885675 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6251ae-9e00-458c-988e-acf3a062b5d8 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:42.895391 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 908/1803] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:37:46.098946 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a528511-4998-4281-bf0e-284237c9f1cf service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:46.108627 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a528511-4998-4281-bf0e-284237c9f1cf service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:46.112643 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f655d15-eb13-4873-bad8-31af34c84b89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:46.113335 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f655d15-eb13-4873-bad8-31af34c84b89 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:46.113637 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f655d15-eb13-4873-bad8-31af34c84b89 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:46.113883 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f655d15-eb13-4873-bad8-31af34c84b89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:46.124185 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f655d15-eb13-4873-bad8-31af34c84b89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:46.125728 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f655d15-eb13-4873-bad8-31af34c84b89 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:46.135342 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 896/1804] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:46 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:37:47.633304 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6251ae-9e00-458c-988e-acf3a062b5d8 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:47.634705 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ca6251ae-9e00-458c-988e-acf3a062b5d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:47.698517 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6251ae-9e00-458c-988e-acf3a062b5d8 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:47.703596 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f7b60-9c99-48d1-a6cc-c271ead7d98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:47.704226 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f7b60-9c99-48d1-a6cc-c271ead7d98b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:47.704488 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f7b60-9c99-48d1-a6cc-c271ead7d98b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:47.704790 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f7b60-9c99-48d1-a6cc-c271ead7d98b admin admin] PATH_INFO: `/v3/users/fa86d4d5527d470e98ca377204463e54` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:47.706131 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407f7b60-9c99-48d1-a6cc-c271ead7d98b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa86d4d5527d470e98ca377204463e54)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:47.708014 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407f7b60-9c99-48d1-a6cc-c271ead7d98b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:47.728577 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:47.753697 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-407f7b60-9c99-48d1-a6cc-c271ead7d98b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa86d4d5527d470e98ca377204463e54'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:47.761086 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-407f7b60-9c99-48d1-a6cc-c271ead7d98b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa86d4d5527d470e98ca377204463e54'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:47.772876 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-407f7b60-9c99-48d1-a6cc-c271ead7d98b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa86d4d5527d470e98ca377204463e54'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:47.780680 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-407f7b60-9c99-48d1-a6cc-c271ead7d98b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa86d4d5527d470e98ca377204463e54'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:47.781427 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-407f7b60-9c99-48d1-a6cc-c271ead7d98b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa86d4d5527d470e98ca377204463e54'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:47.787652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 909/1805] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:37:47 2022] DELETE /identity/v3/users/fa86d4d5527d470e98ca377204463e54 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:37:47.792448 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012eaf81-9050-49f5-8db7-d60de3b09110 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:47.792920 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012eaf81-9050-49f5-8db7-d60de3b09110 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:47.793010 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012eaf81-9050-49f5-8db7-d60de3b09110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:47.819608 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-012eaf81-9050-49f5-8db7-d60de3b09110 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:47.820842 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-012eaf81-9050-49f5-8db7-d60de3b09110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:47.878333 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 897/1806] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:47.888050 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f7b60-9c99-48d1-a6cc-c271ead7d98b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:47.889491 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-407f7b60-9c99-48d1-a6cc-c271ead7d98b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:47.945929 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f7b60-9c99-48d1-a6cc-c271ead7d98b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:47.949525 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28909f-6464-485a-ac1d-e4ae1e8c3e16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:47.950119 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28909f-6464-485a-ac1d-e4ae1e8c3e16 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:47.950392 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28909f-6464-485a-ac1d-e4ae1e8c3e16 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:47.950653 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28909f-6464-485a-ac1d-e4ae1e8c3e16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:47.959655 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd28909f-6464-485a-ac1d-e4ae1e8c3e16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:47.961091 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd28909f-6464-485a-ac1d-e4ae1e8c3e16 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:47.970309 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 910/1807] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:37:48.318192 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012eaf81-9050-49f5-8db7-d60de3b09110 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:48.319707 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-012eaf81-9050-49f5-8db7-d60de3b09110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:48.332438 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012eaf81-9050-49f5-8db7-d60de3b09110 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:48.337655 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0266aa-ae8c-408e-8b80-793d0ce8f647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:48.338291 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0266aa-ae8c-408e-8b80-793d0ce8f647 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:48.338565 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0266aa-ae8c-408e-8b80-793d0ce8f647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:48.338832 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0266aa-ae8c-408e-8b80-793d0ce8f647 admin admin] PATH_INFO: `/v3/projects/7d8a9ad4232a4413822b3ae6ea9613be` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:48.339966 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0266aa-ae8c-408e-8b80-793d0ce8f647 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d8a9ad4232a4413822b3ae6ea9613be)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:48.342425 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0266aa-ae8c-408e-8b80-793d0ce8f647 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:48.391896 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4b0266aa-ae8c-408e-8b80-793d0ce8f647 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d8a9ad4232a4413822b3ae6ea9613be', 'request_id': 'req-4b0266aa-ae8c-408e-8b80-793d0ce8f647'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:48.398371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4b0266aa-ae8c-408e-8b80-793d0ce8f647 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d8a9ad4232a4413822b3ae6ea9613be', 'request_id': 'req-4b0266aa-ae8c-408e-8b80-793d0ce8f647'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:48.404594 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4b0266aa-ae8c-408e-8b80-793d0ce8f647 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d8a9ad4232a4413822b3ae6ea9613be', 'request_id': 'req-4b0266aa-ae8c-408e-8b80-793d0ce8f647'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:48.418499 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4b0266aa-ae8c-408e-8b80-793d0ce8f647 admin admin] The token cache is being invalidate because project 7d8a9ad4232a4413822b3ae6ea9613be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:37:48.419300 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4b0266aa-ae8c-408e-8b80-793d0ce8f647 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:48.419928 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4b0266aa-ae8c-408e-8b80-793d0ce8f647 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:48.421484 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 898/1808] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:37:48 2022] DELETE /identity/v3/projects/7d8a9ad4232a4413822b3ae6ea9613be => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:48.425601 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28909f-6464-485a-ac1d-e4ae1e8c3e16 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:48.435065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28909f-6464-485a-ac1d-e4ae1e8c3e16 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:48.493185 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc5808-d7e9-4e78-8585-b923e8ff652b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:48.493810 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc5808-d7e9-4e78-8585-b923e8ff652b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:48.494088 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc5808-d7e9-4e78-8585-b923e8ff652b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:48.494361 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc5808-d7e9-4e78-8585-b923e8ff652b admin admin] PATH_INFO: `/v3/users/c876052410f348aaa0365598f2aa0a33` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:48.495468 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc5808-d7e9-4e78-8585-b923e8ff652b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c876052410f348aaa0365598f2aa0a33)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:48.497556 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc5808-d7e9-4e78-8585-b923e8ff652b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:48.539004 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:48.545313 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4dfc5808-d7e9-4e78-8585-b923e8ff652b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c876052410f348aaa0365598f2aa0a33'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:48.553432 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4dfc5808-d7e9-4e78-8585-b923e8ff652b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c876052410f348aaa0365598f2aa0a33'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:48.564946 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4dfc5808-d7e9-4e78-8585-b923e8ff652b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c876052410f348aaa0365598f2aa0a33'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:48.572620 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4dfc5808-d7e9-4e78-8585-b923e8ff652b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c876052410f348aaa0365598f2aa0a33'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:48.573368 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4dfc5808-d7e9-4e78-8585-b923e8ff652b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c876052410f348aaa0365598f2aa0a33'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:48.580726 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 911/1809] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:37:48 2022] DELETE /identity/v3/users/c876052410f348aaa0365598f2aa0a33 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:48.853753 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0266aa-ae8c-408e-8b80-793d0ce8f647 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:48.855264 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4b0266aa-ae8c-408e-8b80-793d0ce8f647 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:48.868462 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0266aa-ae8c-408e-8b80-793d0ce8f647 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:48.900406 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:48.929701 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4b728-b9b8-422e-ba34-05ffa19068dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:48.930332 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4b728-b9b8-422e-ba34-05ffa19068dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:48.930612 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4b728-b9b8-422e-ba34-05ffa19068dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:48.930883 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4b728-b9b8-422e-ba34-05ffa19068dc admin admin] PATH_INFO: `/v3/projects/9b4dc0615d014fd4aecac3e4ffeb95aa` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:48.932137 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d4b728-b9b8-422e-ba34-05ffa19068dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b4dc0615d014fd4aecac3e4ffeb95aa)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:48.934600 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d4b728-b9b8-422e-ba34-05ffa19068dc admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:48.979603 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f8d4b728-b9b8-422e-ba34-05ffa19068dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b4dc0615d014fd4aecac3e4ffeb95aa', 'request_id': 'req-f8d4b728-b9b8-422e-ba34-05ffa19068dc'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:48.985539 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f8d4b728-b9b8-422e-ba34-05ffa19068dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b4dc0615d014fd4aecac3e4ffeb95aa', 'request_id': 'req-f8d4b728-b9b8-422e-ba34-05ffa19068dc'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:48.991053 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f8d4b728-b9b8-422e-ba34-05ffa19068dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b4dc0615d014fd4aecac3e4ffeb95aa', 'request_id': 'req-f8d4b728-b9b8-422e-ba34-05ffa19068dc'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:49.003288 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f8d4b728-b9b8-422e-ba34-05ffa19068dc admin admin] The token cache is being invalidate because project 9b4dc0615d014fd4aecac3e4ffeb95aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:37:49.003569 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f8d4b728-b9b8-422e-ba34-05ffa19068dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:49.004174 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f8d4b728-b9b8-422e-ba34-05ffa19068dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:49.005741 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 899/1810] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:37:48 2022] DELETE /identity/v3/projects/9b4dc0615d014fd4aecac3e4ffeb95aa => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:49.010147 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc5808-d7e9-4e78-8585-b923e8ff652b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:49.011692 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4dfc5808-d7e9-4e78-8585-b923e8ff652b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:49.024919 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc5808-d7e9-4e78-8585-b923e8ff652b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:49.080333 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3009f0c4-8e15-4455-939a-5e466b172079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:49.080994 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009f0c4-8e15-4455-939a-5e466b172079 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:49.081270 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009f0c4-8e15-4455-939a-5e466b172079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:49.081541 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009f0c4-8e15-4455-939a-5e466b172079 admin admin] PATH_INFO: `/v3/users/082d9f6b36474f3386173d70b370e3c0` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:49.082673 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3009f0c4-8e15-4455-939a-5e466b172079 admin admin] RBAC: Authorizing `identity:delete_user(user_id=082d9f6b36474f3386173d70b370e3c0)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:49.084615 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3009f0c4-8e15-4455-939a-5e466b172079 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:49.128618 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3009f0c4-8e15-4455-939a-5e466b172079 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '082d9f6b36474f3386173d70b370e3c0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:49.135457 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3009f0c4-8e15-4455-939a-5e466b172079 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '082d9f6b36474f3386173d70b370e3c0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:49.147949 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3009f0c4-8e15-4455-939a-5e466b172079 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '082d9f6b36474f3386173d70b370e3c0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:49.155401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3009f0c4-8e15-4455-939a-5e466b172079 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '082d9f6b36474f3386173d70b370e3c0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:49.156015 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3009f0c4-8e15-4455-939a-5e466b172079 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '082d9f6b36474f3386173d70b370e3c0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:49.162272 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 912/1811] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:37:49 2022] DELETE /identity/v3/users/082d9f6b36474f3386173d70b370e3c0 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:49.438285 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4b728-b9b8-422e-ba34-05ffa19068dc admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:49.439742 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f8d4b728-b9b8-422e-ba34-05ffa19068dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:49.452685 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4b728-b9b8-422e-ba34-05ffa19068dc admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:49.511795 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140e2f90-5737-4a68-ab45-0f27cb991dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:49.512452 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e2f90-5737-4a68-ab45-0f27cb991dfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:49.512769 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e2f90-5737-4a68-ab45-0f27cb991dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:49.513042 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e2f90-5737-4a68-ab45-0f27cb991dfb admin admin] PATH_INFO: `/v3/projects/eae564db29214c0da8f23285668e435b` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:49.514151 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140e2f90-5737-4a68-ab45-0f27cb991dfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=eae564db29214c0da8f23285668e435b)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:49.516084 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140e2f90-5737-4a68-ab45-0f27cb991dfb admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:49.564652 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-140e2f90-5737-4a68-ab45-0f27cb991dfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eae564db29214c0da8f23285668e435b', 'request_id': 'req-140e2f90-5737-4a68-ab45-0f27cb991dfb'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:49.570908 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-140e2f90-5737-4a68-ab45-0f27cb991dfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eae564db29214c0da8f23285668e435b', 'request_id': 'req-140e2f90-5737-4a68-ab45-0f27cb991dfb'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:49.576848 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-140e2f90-5737-4a68-ab45-0f27cb991dfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eae564db29214c0da8f23285668e435b', 'request_id': 'req-140e2f90-5737-4a68-ab45-0f27cb991dfb'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:49.578487 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:49.594620 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-140e2f90-5737-4a68-ab45-0f27cb991dfb admin admin] The token cache is being invalidate because project eae564db29214c0da8f23285668e435b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:37:49.594927 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-140e2f90-5737-4a68-ab45-0f27cb991dfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:49.595619 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-140e2f90-5737-4a68-ab45-0f27cb991dfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:49.597182 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 900/1812] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:37:49 2022] DELETE /identity/v3/projects/eae564db29214c0da8f23285668e435b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:49.602067 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32430455-8ee9-4cfd-b504-0fb781e14a23 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:49.602289 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32430455-8ee9-4cfd-b504-0fb781e14a23 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:49.602560 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32430455-8ee9-4cfd-b504-0fb781e14a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:49.624252 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-32430455-8ee9-4cfd-b504-0fb781e14a23 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:49.625506 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-32430455-8ee9-4cfd-b504-0fb781e14a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:49.670252 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:49.698646 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 913/1813] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:49.706845 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3a405-b33d-4828-832c-ea04e9e167a1 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:49.707131 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3a405-b33d-4828-832c-ea04e9e167a1 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:49.707425 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3a405-b33d-4828-832c-ea04e9e167a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:49.727214 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-e0c3a405-b33d-4828-832c-ea04e9e167a1 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:49.728410 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e0c3a405-b33d-4828-832c-ea04e9e167a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:49.736904 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 901/1814] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:49.740962 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32430455-8ee9-4cfd-b504-0fb781e14a23 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:49.751438 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32430455-8ee9-4cfd-b504-0fb781e14a23 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:49.756187 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4542b741-8d8e-4cbd-9d7a-38568c5eac40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:49.756961 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4542b741-8d8e-4cbd-9d7a-38568c5eac40 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:49.757238 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4542b741-8d8e-4cbd-9d7a-38568c5eac40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:49.757501 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4542b741-8d8e-4cbd-9d7a-38568c5eac40 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:49.758125 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4542b741-8d8e-4cbd-9d7a-38568c5eac40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:37:49.758421 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4542b741-8d8e-4cbd-9d7a-38568c5eac40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:49.759995 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4542b741-8d8e-4cbd-9d7a-38568c5eac40 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:49.771495 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 914/1815] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:37:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:37:49.775929 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c3a405-b33d-4828-832c-ea04e9e167a1 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:49.785564 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c3a405-b33d-4828-832c-ea04e9e167a1 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:49.790204 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d582d98e-5d89-4d73-af08-b25336265926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:49.790798 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582d98e-5d89-4d73-af08-b25336265926 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:49.791073 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582d98e-5d89-4d73-af08-b25336265926 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:49.791339 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582d98e-5d89-4d73-af08-b25336265926 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:49.791958 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d582d98e-5d89-4d73-af08-b25336265926 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:49.794504 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d582d98e-5d89-4d73-af08-b25336265926 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:49.821642 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 902/1816] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:37:49 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:37:49.825851 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4542b741-8d8e-4cbd-9d7a-38568c5eac40 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:49.835844 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4542b741-8d8e-4cbd-9d7a-38568c5eac40 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:49.890729 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38efd57d-a446-4acd-b836-e9279fb6d577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:49.891342 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38efd57d-a446-4acd-b836-e9279fb6d577 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:49.891623 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38efd57d-a446-4acd-b836-e9279fb6d577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:49.891893 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38efd57d-a446-4acd-b836-e9279fb6d577 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:49.892552 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38efd57d-a446-4acd-b836-e9279fb6d577 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:49.894523 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38efd57d-a446-4acd-b836-e9279fb6d577 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:49.921767 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 915/1817] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:37:49 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:37:49.924058 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d582d98e-5d89-4d73-af08-b25336265926 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:49.939426 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d582d98e-5d89-4d73-af08-b25336265926 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:49.944126 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f3db58-7af0-4b5a-a8f8-5e91ec178901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:49.944748 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3db58-7af0-4b5a-a8f8-5e91ec178901 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:49.945052 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3db58-7af0-4b5a-a8f8-5e91ec178901 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:49.945319 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3db58-7af0-4b5a-a8f8-5e91ec178901 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:49.946077 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f3db58-7af0-4b5a-a8f8-5e91ec178901 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:49.947666 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f3db58-7af0-4b5a-a8f8-5e91ec178901 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:49.958037 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 903/1818] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:49.962181 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38efd57d-a446-4acd-b836-e9279fb6d577 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:49.972509 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38efd57d-a446-4acd-b836-e9279fb6d577 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:49.977092 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492ec14d-4580-4361-8501-2a73e4fb7ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:49.977683 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ec14d-4580-4361-8501-2a73e4fb7ae5 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:49.977963 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ec14d-4580-4361-8501-2a73e4fb7ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:49.978231 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ec14d-4580-4361-8501-2a73e4fb7ae5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:49.978859 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492ec14d-4580-4361-8501-2a73e4fb7ae5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:49.980417 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492ec14d-4580-4361-8501-2a73e4fb7ae5 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:49.991508 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 916/1819] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:49.995403 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f3db58-7af0-4b5a-a8f8-5e91ec178901 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:50.005244 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f3db58-7af0-4b5a-a8f8-5e91ec178901 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:50.010214 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046bf3d8-fc69-48ed-a3ef-26f63cbc41e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:50.010800 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046bf3d8-fc69-48ed-a3ef-26f63cbc41e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:50.011074 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046bf3d8-fc69-48ed-a3ef-26f63cbc41e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:50.011343 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046bf3d8-fc69-48ed-a3ef-26f63cbc41e4 admin admin] PATH_INFO: `/v3/projects/f50221b5030244819ca4809e31e90404/users/0bd43bca166a4713a4cf76450d02f749/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:50.026131 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046bf3d8-fc69-48ed-a3ef-26f63cbc41e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f50221b5030244819ca4809e31e90404, user_id=0bd43bca166a4713a4cf76450d02f749, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:50.029485 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046bf3d8-fc69-48ed-a3ef-26f63cbc41e4 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:50.041566 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 904/1820] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:37:49 2022] PUT /identity/v3/projects/f50221b5030244819ca4809e31e90404/users/0bd43bca166a4713a4cf76450d02f749/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:37:50.051635 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abe024e-1498-4646-90c5-233902d0ac1a None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:50.052103 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abe024e-1498-4646-90c5-233902d0ac1a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:50.052211 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abe024e-1498-4646-90c5-233902d0ac1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:50.080534 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-4abe024e-1498-4646-90c5-233902d0ac1a None None] MFA Rules not processed for user `0bd43bca166a4713a4cf76450d02f749`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:50.081770 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4abe024e-1498-4646-90c5-233902d0ac1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:50.147684 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 917/1821] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:50.158733 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046bf3d8-fc69-48ed-a3ef-26f63cbc41e4 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:50.160399 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-046bf3d8-fc69-48ed-a3ef-26f63cbc41e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:50.219326 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046bf3d8-fc69-48ed-a3ef-26f63cbc41e4 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:50.223063 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e705c3-0413-46f8-8077-ac277603a4d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:50.223690 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e705c3-0413-46f8-8077-ac277603a4d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:50.223942 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e705c3-0413-46f8-8077-ac277603a4d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:50.224199 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e705c3-0413-46f8-8077-ac277603a4d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:50.232351 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e705c3-0413-46f8-8077-ac277603a4d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:50.233819 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e705c3-0413-46f8-8077-ac277603a4d2 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:50.243468 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 905/1822] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:37:52.205718 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abe024e-1498-4646-90c5-233902d0ac1a None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:52.207143 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4abe024e-1498-4646-90c5-233902d0ac1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:52.243407 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:52.281162 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abe024e-1498-4646-90c5-233902d0ac1a None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:52.285990 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc470db-879a-4d9c-b5da-355f8da58f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:52.286628 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc470db-879a-4d9c-b5da-355f8da58f8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:52.286909 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc470db-879a-4d9c-b5da-355f8da58f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:52.287548 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc470db-879a-4d9c-b5da-355f8da58f8a admin admin] PATH_INFO: `/v3/users/37f0f7cc992e447a9ea08e711e226e4f` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:52.288443 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc470db-879a-4d9c-b5da-355f8da58f8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=37f0f7cc992e447a9ea08e711e226e4f)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:52.290303 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc470db-879a-4d9c-b5da-355f8da58f8a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:52.353741 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8cc470db-879a-4d9c-b5da-355f8da58f8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37f0f7cc992e447a9ea08e711e226e4f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:52.365031 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8cc470db-879a-4d9c-b5da-355f8da58f8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37f0f7cc992e447a9ea08e711e226e4f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:52.379935 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8cc470db-879a-4d9c-b5da-355f8da58f8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37f0f7cc992e447a9ea08e711e226e4f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:52.394136 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8cc470db-879a-4d9c-b5da-355f8da58f8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37f0f7cc992e447a9ea08e711e226e4f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:52.394760 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8cc470db-879a-4d9c-b5da-355f8da58f8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37f0f7cc992e447a9ea08e711e226e4f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:52.409054 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 918/1823] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:37:52 2022] DELETE /identity/v3/users/37f0f7cc992e447a9ea08e711e226e4f => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:52.416669 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f68f1d-c257-4ab3-a31c-d4a70e78f31c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:52.417205 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f68f1d-c257-4ab3-a31c-d4a70e78f31c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:52.417205 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f68f1d-c257-4ab3-a31c-d4a70e78f31c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:52.441324 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-09f68f1d-c257-4ab3-a31c-d4a70e78f31c None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:52.442814 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-09f68f1d-c257-4ab3-a31c-d4a70e78f31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:52.513219 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 906/1824] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:37:52.523202 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc470db-879a-4d9c-b5da-355f8da58f8a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:52.524769 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8cc470db-879a-4d9c-b5da-355f8da58f8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:52.586931 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc470db-879a-4d9c-b5da-355f8da58f8a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:52.591320 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9679851a-c704-475e-9723-fda4a191e838 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:52.591929 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9679851a-c704-475e-9723-fda4a191e838 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:52.592205 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9679851a-c704-475e-9723-fda4a191e838 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:52.592474 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9679851a-c704-475e-9723-fda4a191e838 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:52.601694 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9679851a-c704-475e-9723-fda4a191e838 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:52.603195 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9679851a-c704-475e-9723-fda4a191e838 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:52.612983 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 919/1825] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:37:52.897738 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f68f1d-c257-4ab3-a31c-d4a70e78f31c None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:52.899159 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-09f68f1d-c257-4ab3-a31c-d4a70e78f31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:52.912474 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f68f1d-c257-4ab3-a31c-d4a70e78f31c None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:52.918894 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f27fa2c-fa50-4e96-915c-b027739bacd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:52.919664 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27fa2c-fa50-4e96-915c-b027739bacd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:52.920054 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27fa2c-fa50-4e96-915c-b027739bacd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:52.920477 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27fa2c-fa50-4e96-915c-b027739bacd7 admin admin] PATH_INFO: `/v3/projects/7529b42d0cd24f23a9023a9728bbf1c6` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:52.921974 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f27fa2c-fa50-4e96-915c-b027739bacd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7529b42d0cd24f23a9023a9728bbf1c6)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:52.923898 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f27fa2c-fa50-4e96-915c-b027739bacd7 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:52.962674 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:INSERT=3,keystone:DELETE=2,keystone:SELECT=95 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:52.983686 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0f27fa2c-fa50-4e96-915c-b027739bacd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7529b42d0cd24f23a9023a9728bbf1c6', 'request_id': 'req-0f27fa2c-fa50-4e96-915c-b027739bacd7'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:52.989952 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0f27fa2c-fa50-4e96-915c-b027739bacd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7529b42d0cd24f23a9023a9728bbf1c6', 'request_id': 'req-0f27fa2c-fa50-4e96-915c-b027739bacd7'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:52.995695 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0f27fa2c-fa50-4e96-915c-b027739bacd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7529b42d0cd24f23a9023a9728bbf1c6', 'request_id': 'req-0f27fa2c-fa50-4e96-915c-b027739bacd7'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:53.008393 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0f27fa2c-fa50-4e96-915c-b027739bacd7 admin admin] The token cache is being invalidate because project 7529b42d0cd24f23a9023a9728bbf1c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:37:53.008907 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0f27fa2c-fa50-4e96-915c-b027739bacd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:53.009607 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0f27fa2c-fa50-4e96-915c-b027739bacd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:53.011093 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 907/1826] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:37:52 2022] DELETE /identity/v3/projects/7529b42d0cd24f23a9023a9728bbf1c6 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:53.015780 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9679851a-c704-475e-9723-fda4a191e838 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:53.026514 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9679851a-c704-475e-9723-fda4a191e838 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:53.085920 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:53.095324 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1579d3-cdff-490f-b00f-579be6398996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:53.096348 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1579d3-cdff-490f-b00f-579be6398996 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:53.097111 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1579d3-cdff-490f-b00f-579be6398996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:53.097575 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1579d3-cdff-490f-b00f-579be6398996 admin admin] PATH_INFO: `/v3/users/411e5dd885344de7b7a88b2a9662433f` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:53.098846 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1579d3-cdff-490f-b00f-579be6398996 admin admin] RBAC: Authorizing `identity:delete_user(user_id=411e5dd885344de7b7a88b2a9662433f)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:53.101779 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1579d3-cdff-490f-b00f-579be6398996 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:53.166670 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-fa1579d3-cdff-490f-b00f-579be6398996 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '411e5dd885344de7b7a88b2a9662433f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:53.177939 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-fa1579d3-cdff-490f-b00f-579be6398996 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '411e5dd885344de7b7a88b2a9662433f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:53.203533 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-fa1579d3-cdff-490f-b00f-579be6398996 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '411e5dd885344de7b7a88b2a9662433f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:53.218187 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-fa1579d3-cdff-490f-b00f-579be6398996 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '411e5dd885344de7b7a88b2a9662433f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:53.219065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-fa1579d3-cdff-490f-b00f-579be6398996 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '411e5dd885344de7b7a88b2a9662433f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:53.227306 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 920/1827] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:37:53 2022] DELETE /identity/v3/users/411e5dd885344de7b7a88b2a9662433f => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:53.559385 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f27fa2c-fa50-4e96-915c-b027739bacd7 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:53.563150 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0f27fa2c-fa50-4e96-915c-b027739bacd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:53.581476 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f27fa2c-fa50-4e96-915c-b027739bacd7 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:53.674268 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0227bc-6922-47f6-9dff-5fbfd8743ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:53.674268 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0227bc-6922-47f6-9dff-5fbfd8743ab9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:53.674268 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0227bc-6922-47f6-9dff-5fbfd8743ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:53.674268 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0227bc-6922-47f6-9dff-5fbfd8743ab9 admin admin] PATH_INFO: `/v3/projects/fb8fcb81ff684d83a73b114978b1d199` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:53.674613 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0227bc-6922-47f6-9dff-5fbfd8743ab9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb8fcb81ff684d83a73b114978b1d199)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:53.684692 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0227bc-6922-47f6-9dff-5fbfd8743ab9 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:53.736084 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-db0227bc-6922-47f6-9dff-5fbfd8743ab9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb8fcb81ff684d83a73b114978b1d199', 'request_id': 'req-db0227bc-6922-47f6-9dff-5fbfd8743ab9'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:53.742109 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-db0227bc-6922-47f6-9dff-5fbfd8743ab9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb8fcb81ff684d83a73b114978b1d199', 'request_id': 'req-db0227bc-6922-47f6-9dff-5fbfd8743ab9'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:53.748571 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-db0227bc-6922-47f6-9dff-5fbfd8743ab9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb8fcb81ff684d83a73b114978b1d199', 'request_id': 'req-db0227bc-6922-47f6-9dff-5fbfd8743ab9'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:53.761188 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-db0227bc-6922-47f6-9dff-5fbfd8743ab9 admin admin] The token cache is being invalidate because project fb8fcb81ff684d83a73b114978b1d199 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:37:53.761466 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-db0227bc-6922-47f6-9dff-5fbfd8743ab9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:53.762113 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-db0227bc-6922-47f6-9dff-5fbfd8743ab9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:53.763670 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 908/1828] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:37:53 2022] DELETE /identity/v3/projects/fb8fcb81ff684d83a73b114978b1d199 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:53.767772 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1579d3-cdff-490f-b00f-579be6398996 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:53.769283 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-fa1579d3-cdff-490f-b00f-579be6398996 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:53.781872 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1579d3-cdff-490f-b00f-579be6398996 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:53.838966 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c31c939-54ac-4011-a8e6-88674a1a3a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:53.839597 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c31c939-54ac-4011-a8e6-88674a1a3a4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:53.839880 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c31c939-54ac-4011-a8e6-88674a1a3a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:53.840145 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c31c939-54ac-4011-a8e6-88674a1a3a4a admin admin] PATH_INFO: `/v3/users/b4d1f626514049f7839fe241d466d98f` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:53.841309 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c31c939-54ac-4011-a8e6-88674a1a3a4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4d1f626514049f7839fe241d466d98f)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:53.843358 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c31c939-54ac-4011-a8e6-88674a1a3a4a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:53.890063 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4c31c939-54ac-4011-a8e6-88674a1a3a4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4d1f626514049f7839fe241d466d98f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:53.896890 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4c31c939-54ac-4011-a8e6-88674a1a3a4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4d1f626514049f7839fe241d466d98f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:53.908741 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4c31c939-54ac-4011-a8e6-88674a1a3a4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4d1f626514049f7839fe241d466d98f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:53.911149 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:53.916815 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4c31c939-54ac-4011-a8e6-88674a1a3a4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4d1f626514049f7839fe241d466d98f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:53.917943 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4c31c939-54ac-4011-a8e6-88674a1a3a4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4d1f626514049f7839fe241d466d98f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:53.925335 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 921/1829] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:37:53 2022] DELETE /identity/v3/users/b4d1f626514049f7839fe241d466d98f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:54.213519 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0227bc-6922-47f6-9dff-5fbfd8743ab9 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:54.215633 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-db0227bc-6922-47f6-9dff-5fbfd8743ab9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:54.232449 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0227bc-6922-47f6-9dff-5fbfd8743ab9 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:54.296786 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:54.314679 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc6123-c22e-4f20-8846-11352b6bf45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:54.315550 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc6123-c22e-4f20-8846-11352b6bf45f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:54.315829 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc6123-c22e-4f20-8846-11352b6bf45f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:54.316103 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc6123-c22e-4f20-8846-11352b6bf45f admin admin] PATH_INFO: `/v3/projects/38d5a8cd58ec459283642d104dc2aa5a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:54.317357 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffc6123-c22e-4f20-8846-11352b6bf45f admin admin] RBAC: Authorizing `identity:delete_project(project_id=38d5a8cd58ec459283642d104dc2aa5a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:54.319321 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffc6123-c22e-4f20-8846-11352b6bf45f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:54.373602 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-7ffc6123-c22e-4f20-8846-11352b6bf45f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38d5a8cd58ec459283642d104dc2aa5a', 'request_id': 'req-7ffc6123-c22e-4f20-8846-11352b6bf45f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:54.381407 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-7ffc6123-c22e-4f20-8846-11352b6bf45f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38d5a8cd58ec459283642d104dc2aa5a', 'request_id': 'req-7ffc6123-c22e-4f20-8846-11352b6bf45f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:54.387814 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-7ffc6123-c22e-4f20-8846-11352b6bf45f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38d5a8cd58ec459283642d104dc2aa5a', 'request_id': 'req-7ffc6123-c22e-4f20-8846-11352b6bf45f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:54.412281 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-7ffc6123-c22e-4f20-8846-11352b6bf45f admin admin] The token cache is being invalidate because project 38d5a8cd58ec459283642d104dc2aa5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:37:54.412608 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-7ffc6123-c22e-4f20-8846-11352b6bf45f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:54.413270 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-7ffc6123-c22e-4f20-8846-11352b6bf45f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:54.415014 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 909/1830] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:37:54 2022] DELETE /identity/v3/projects/38d5a8cd58ec459283642d104dc2aa5a => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:54.420338 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c8abf-817a-46ef-bebe-a32aed4645b7 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:54.421407 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c8abf-817a-46ef-bebe-a32aed4645b7 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:54.421911 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c8abf-817a-46ef-bebe-a32aed4645b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:54.443748 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-299c8abf-817a-46ef-bebe-a32aed4645b7 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:54.445032 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-299c8abf-817a-46ef-bebe-a32aed4645b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:54.521720 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 922/1831] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:54.530095 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a9bad-537a-4934-a375-86d937e2f259 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:54.530211 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a9bad-537a-4934-a375-86d937e2f259 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:54.530618 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a9bad-537a-4934-a375-86d937e2f259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:54.550864 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-ec8a9bad-537a-4934-a375-86d937e2f259 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:54.552045 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ec8a9bad-537a-4934-a375-86d937e2f259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:54.560827 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 910/1832] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:54.565984 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c8abf-817a-46ef-bebe-a32aed4645b7 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:54.577089 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c8abf-817a-46ef-bebe-a32aed4645b7 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:54.584418 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50621f61-6da3-44c4-b809-27deff9bdedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:54.585125 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50621f61-6da3-44c4-b809-27deff9bdedc admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:54.585416 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50621f61-6da3-44c4-b809-27deff9bdedc admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:54.585682 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50621f61-6da3-44c4-b809-27deff9bdedc admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:54.586322 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50621f61-6da3-44c4-b809-27deff9bdedc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:37:54.586624 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50621f61-6da3-44c4-b809-27deff9bdedc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:54.588354 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50621f61-6da3-44c4-b809-27deff9bdedc admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:54.602427 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 923/1833] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:37:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:37:54.607047 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8a9bad-537a-4934-a375-86d937e2f259 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:54.619126 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8a9bad-537a-4934-a375-86d937e2f259 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:54.630340 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7afeaf-24cf-497d-aedd-3f48be1a59ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:54.633613 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7afeaf-24cf-497d-aedd-3f48be1a59ab admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:54.633982 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7afeaf-24cf-497d-aedd-3f48be1a59ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:54.634330 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7afeaf-24cf-497d-aedd-3f48be1a59ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:54.635057 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7afeaf-24cf-497d-aedd-3f48be1a59ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:54.637776 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7afeaf-24cf-497d-aedd-3f48be1a59ab admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:54.669983 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 911/1834] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:37:54 2022] POST /identity/v3/projects => generated 371 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:37:54.674253 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50621f61-6da3-44c4-b809-27deff9bdedc admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:54.688712 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50621f61-6da3-44c4-b809-27deff9bdedc admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:54.789850 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a8609-a892-4934-a707-951cfab5c5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:54.790654 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a8609-a892-4934-a707-951cfab5c5c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:54.791093 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a8609-a892-4934-a707-951cfab5c5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:54.791367 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a8609-a892-4934-a707-951cfab5c5c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:54.792005 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192a8609-a892-4934-a707-951cfab5c5c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:54.793972 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192a8609-a892-4934-a707-951cfab5c5c0 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:54.821304 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 924/1835] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:37:54 2022] POST /identity/v3/users => generated 286 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:37:54.825881 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7afeaf-24cf-497d-aedd-3f48be1a59ab admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:54.838533 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7afeaf-24cf-497d-aedd-3f48be1a59ab admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:54.847544 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f997392-f7ed-4318-aa65-2f97ba636616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:54.849306 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f997392-f7ed-4318-aa65-2f97ba636616 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:54.849757 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f997392-f7ed-4318-aa65-2f97ba636616 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:54.850187 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f997392-f7ed-4318-aa65-2f97ba636616 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:54.850717 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f997392-f7ed-4318-aa65-2f97ba636616 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:54.852395 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f997392-f7ed-4318-aa65-2f97ba636616 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:54.863873 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 912/1836] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:54.867739 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a8609-a892-4934-a707-951cfab5c5c0 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:54.878459 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a8609-a892-4934-a707-951cfab5c5c0 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:54.883279 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a07381-05ed-487b-95a1-3c72526c2d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:54.883916 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a07381-05ed-487b-95a1-3c72526c2d21 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:54.884294 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a07381-05ed-487b-95a1-3c72526c2d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:54.884697 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a07381-05ed-487b-95a1-3c72526c2d21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:54.885436 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a07381-05ed-487b-95a1-3c72526c2d21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:54.887723 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a07381-05ed-487b-95a1-3c72526c2d21 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:54.900584 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 925/1837] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 09:37:54.904643 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f997392-f7ed-4318-aa65-2f97ba636616 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:54.915016 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f997392-f7ed-4318-aa65-2f97ba636616 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:54.919757 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaad3a9e-b76f-4a9a-99a1-cac65c2e8078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:54.920406 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad3a9e-b76f-4a9a-99a1-cac65c2e8078 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:54.920725 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad3a9e-b76f-4a9a-99a1-cac65c2e8078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:54.920988 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad3a9e-b76f-4a9a-99a1-cac65c2e8078 admin admin] PATH_INFO: `/v3/projects/e694ff25532a41bfac762f1e11e3869c/users/c625f65762224ff9a799243ef0b527fc/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:54.936734 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaad3a9e-b76f-4a9a-99a1-cac65c2e8078 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e694ff25532a41bfac762f1e11e3869c, user_id=c625f65762224ff9a799243ef0b527fc, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:54.939174 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaad3a9e-b76f-4a9a-99a1-cac65c2e8078 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:54.955809 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 913/1838] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:37:54 2022] PUT /identity/v3/projects/e694ff25532a41bfac762f1e11e3869c/users/c625f65762224ff9a799243ef0b527fc/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:55.031522 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6875d-b98c-4fea-879d-3284cbb5df06 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:55.031806 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6875d-b98c-4fea-879d-3284cbb5df06 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:55.032078 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6875d-b98c-4fea-879d-3284cbb5df06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:55.065543 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-fff6875d-b98c-4fea-879d-3284cbb5df06 None None] MFA Rules not processed for user `c625f65762224ff9a799243ef0b527fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:55.066806 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-fff6875d-b98c-4fea-879d-3284cbb5df06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:55.129364 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:55.151726 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 926/1839] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:55.162349 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaad3a9e-b76f-4a9a-99a1-cac65c2e8078 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:55.164292 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-eaad3a9e-b76f-4a9a-99a1-cac65c2e8078 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:55.231512 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaad3a9e-b76f-4a9a-99a1-cac65c2e8078 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:55.235981 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f80da21-4d01-459a-b1e3-829dc4f929bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:55.236239 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80da21-4d01-459a-b1e3-829dc4f929bc service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:55.236538 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80da21-4d01-459a-b1e3-829dc4f929bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:55.236813 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80da21-4d01-459a-b1e3-829dc4f929bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:55.247258 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f80da21-4d01-459a-b1e3-829dc4f929bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:55.248591 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f80da21-4d01-459a-b1e3-829dc4f929bc service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:55.258681 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 914/1840] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:37:55 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:37:57.820632 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ab08a-3d95-4290-b303-fcd7ddda795d None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:57.821170 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ab08a-3d95-4290-b303-fcd7ddda795d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:57.821170 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ab08a-3d95-4290-b303-fcd7ddda795d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:57.842605 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-036ab08a-3d95-4290-b303-fcd7ddda795d None None] MFA Rules not processed for user `c625f65762224ff9a799243ef0b527fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:57.843910 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-036ab08a-3d95-4290-b303-fcd7ddda795d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:57.851104 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 927/1841] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:58.845570 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f80da21-4d01-459a-b1e3-829dc4f929bc service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:58.847040 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3f80da21-4d01-459a-b1e3-829dc4f929bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:58.873846 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:58.924781 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f80da21-4d01-459a-b1e3-829dc4f929bc service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:58.929574 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7270558b-0432-42af-934b-aaa1a2bb87d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:58.930209 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270558b-0432-42af-934b-aaa1a2bb87d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:58.930520 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270558b-0432-42af-934b-aaa1a2bb87d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:58.930790 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270558b-0432-42af-934b-aaa1a2bb87d6 admin admin] PATH_INFO: `/v3/users/6b8fa2a89eee41d09977d7c85c49707b` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:58.932673 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7270558b-0432-42af-934b-aaa1a2bb87d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b8fa2a89eee41d09977d7c85c49707b)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:58.933991 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7270558b-0432-42af-934b-aaa1a2bb87d6 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:58.985443 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-7270558b-0432-42af-934b-aaa1a2bb87d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b8fa2a89eee41d09977d7c85c49707b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:58.993045 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-7270558b-0432-42af-934b-aaa1a2bb87d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b8fa2a89eee41d09977d7c85c49707b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:59.006567 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-7270558b-0432-42af-934b-aaa1a2bb87d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b8fa2a89eee41d09977d7c85c49707b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:59.017512 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-7270558b-0432-42af-934b-aaa1a2bb87d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b8fa2a89eee41d09977d7c85c49707b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:59.018098 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-7270558b-0432-42af-934b-aaa1a2bb87d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b8fa2a89eee41d09977d7c85c49707b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:59.024750 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 915/1842] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:37:58 2022] DELETE /identity/v3/users/6b8fa2a89eee41d09977d7c85c49707b => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:59.209567 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ab08a-3d95-4290-b303-fcd7ddda795d None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:59.221933 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ab08a-3d95-4290-b303-fcd7ddda795d None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:59.305225 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7554058-f765-40bd-b36e-2aac702cbd66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:59.305875 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7554058-f765-40bd-b36e-2aac702cbd66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:59.306151 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7554058-f765-40bd-b36e-2aac702cbd66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:59.306421 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7554058-f765-40bd-b36e-2aac702cbd66 admin admin] PATH_INFO: `/v3/projects/a417696418e34d1e9abaa74e3c5627e1` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:59.307527 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7554058-f765-40bd-b36e-2aac702cbd66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a417696418e34d1e9abaa74e3c5627e1)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:59.309483 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7554058-f765-40bd-b36e-2aac702cbd66 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:59.357336 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b7554058-f765-40bd-b36e-2aac702cbd66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a417696418e34d1e9abaa74e3c5627e1', 'request_id': 'req-b7554058-f765-40bd-b36e-2aac702cbd66'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:59.363164 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b7554058-f765-40bd-b36e-2aac702cbd66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a417696418e34d1e9abaa74e3c5627e1', 'request_id': 'req-b7554058-f765-40bd-b36e-2aac702cbd66'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:59.370916 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b7554058-f765-40bd-b36e-2aac702cbd66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a417696418e34d1e9abaa74e3c5627e1', 'request_id': 'req-b7554058-f765-40bd-b36e-2aac702cbd66'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:59.382751 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b7554058-f765-40bd-b36e-2aac702cbd66 admin admin] The token cache is being invalidate because project a417696418e34d1e9abaa74e3c5627e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:37:59.383038 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b7554058-f765-40bd-b36e-2aac702cbd66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:59.383678 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b7554058-f765-40bd-b36e-2aac702cbd66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:37:59.385339 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 928/1843] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:37:59 2022] DELETE /identity/v3/projects/a417696418e34d1e9abaa74e3c5627e1 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:37:59.390911 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b636ab-292b-457b-a559-a3ec3fab119b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:59.391283 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b636ab-292b-457b-a559-a3ec3fab119b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:59.391629 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b636ab-292b-457b-a559-a3ec3fab119b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:59.414047 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-e8b636ab-292b-457b-a559-a3ec3fab119b None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:59.415554 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e8b636ab-292b-457b-a559-a3ec3fab119b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:59.481405 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 916/1844] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:59.489391 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace18111-a441-4e2a-a45c-ddc9e5550d61 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:59.489786 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace18111-a441-4e2a-a45c-ddc9e5550d61 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:59.490291 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace18111-a441-4e2a-a45c-ddc9e5550d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:59.521542 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-ace18111-a441-4e2a-a45c-ddc9e5550d61 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:59.522818 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ace18111-a441-4e2a-a45c-ddc9e5550d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:37:59.531515 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 929/1845] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:37:59.535479 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b636ab-292b-457b-a559-a3ec3fab119b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:59.546459 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b636ab-292b-457b-a559-a3ec3fab119b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:59.551500 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e4c127-984b-4024-b55e-d026c6ca3f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:59.552101 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4c127-984b-4024-b55e-d026c6ca3f49 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:59.552376 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4c127-984b-4024-b55e-d026c6ca3f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:59.552816 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4c127-984b-4024-b55e-d026c6ca3f49 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:59.553416 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e4c127-984b-4024-b55e-d026c6ca3f49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:37:59.553715 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e4c127-984b-4024-b55e-d026c6ca3f49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:59.555454 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e4c127-984b-4024-b55e-d026c6ca3f49 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:59.569815 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 917/1846] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:37:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:37:59.575056 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace18111-a441-4e2a-a45c-ddc9e5550d61 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:59.585341 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace18111-a441-4e2a-a45c-ddc9e5550d61 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:59.590783 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc65d4d-a881-4f11-981c-50fc1f2afe39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:59.591499 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc65d4d-a881-4f11-981c-50fc1f2afe39 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:59.591844 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc65d4d-a881-4f11-981c-50fc1f2afe39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:59.592185 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc65d4d-a881-4f11-981c-50fc1f2afe39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:59.592918 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc65d4d-a881-4f11-981c-50fc1f2afe39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:59.594861 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc65d4d-a881-4f11-981c-50fc1f2afe39 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:59.628733 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 930/1847] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:37:59 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:37:59.633394 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e4c127-984b-4024-b55e-d026c6ca3f49 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:59.651008 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e4c127-984b-4024-b55e-d026c6ca3f49 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:59.677017 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:37:59.750941 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f30d4-0b0d-47d5-b382-6105e32e4f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:59.751560 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f30d4-0b0d-47d5-b382-6105e32e4f85 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:59.751833 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f30d4-0b0d-47d5-b382-6105e32e4f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:59.752184 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f30d4-0b0d-47d5-b382-6105e32e4f85 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:59.753052 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18f30d4-0b0d-47d5-b382-6105e32e4f85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:59.754657 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18f30d4-0b0d-47d5-b382-6105e32e4f85 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:59.797915 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 918/1848] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:37:59 2022] POST /identity/v3/users => generated 292 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:37:59.799414 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc65d4d-a881-4f11-981c-50fc1f2afe39 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:59.810882 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc65d4d-a881-4f11-981c-50fc1f2afe39 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:59.815508 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254c589d-35b2-4b9c-a434-f14f91c7383c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:59.816119 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c589d-35b2-4b9c-a434-f14f91c7383c admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:59.816404 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c589d-35b2-4b9c-a434-f14f91c7383c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:59.816707 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c589d-35b2-4b9c-a434-f14f91c7383c admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:59.817342 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254c589d-35b2-4b9c-a434-f14f91c7383c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:59.818916 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254c589d-35b2-4b9c-a434-f14f91c7383c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:59.832321 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 931/1849] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:59.833536 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f30d4-0b0d-47d5-b382-6105e32e4f85 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:59.844231 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f30d4-0b0d-47d5-b382-6105e32e4f85 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:59.849639 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cc96a3-e45a-444d-ae05-da4b938a29f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:59.850397 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cc96a3-e45a-444d-ae05-da4b938a29f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:59.850686 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cc96a3-e45a-444d-ae05-da4b938a29f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:59.850961 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cc96a3-e45a-444d-ae05-da4b938a29f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:59.851600 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cc96a3-e45a-444d-ae05-da4b938a29f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:59.853272 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cc96a3-e45a-444d-ae05-da4b938a29f8 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:59.864203 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 919/1850] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:37:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:37:59.868718 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254c589d-35b2-4b9c-a434-f14f91c7383c admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:37:59.880412 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254c589d-35b2-4b9c-a434-f14f91c7383c admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:37:59.885798 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3e9241-d64e-4609-98a1-c8ee53258e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:37:59.886460 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e9241-d64e-4609-98a1-c8ee53258e77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:59.887136 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e9241-d64e-4609-98a1-c8ee53258e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:59.887136 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e9241-d64e-4609-98a1-c8ee53258e77 admin admin] PATH_INFO: `/v3/projects/df84eaa566a24d3e8d25c1065e8c09fa/users/f7d2c6275669404db608d714baeb0065/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:59.910917 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3e9241-d64e-4609-98a1-c8ee53258e77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df84eaa566a24d3e8d25c1065e8c09fa, user_id=f7d2c6275669404db608d714baeb0065, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:37:59.913931 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3e9241-d64e-4609-98a1-c8ee53258e77 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:37:59.937770 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 932/1851] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:37:59 2022] PUT /identity/v3/projects/df84eaa566a24d3e8d25c1065e8c09fa/users/f7d2c6275669404db608d714baeb0065/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:37:59.942778 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad77659-f654-4689-bd97-495bc038732a None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:37:59.943232 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad77659-f654-4689-bd97-495bc038732a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:37:59.943515 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad77659-f654-4689-bd97-495bc038732a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:37:59.965696 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-7ad77659-f654-4689-bd97-495bc038732a None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:37:59.966943 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7ad77659-f654-4689-bd97-495bc038732a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:00.040938 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 920/1852] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:37:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:38:00.051513 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3e9241-d64e-4609-98a1-c8ee53258e77 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:00.053225 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3c3e9241-d64e-4609-98a1-c8ee53258e77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:00.064608 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:38:00.116125 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3e9241-d64e-4609-98a1-c8ee53258e77 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:00.120687 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ce90d-fb9f-4c25-a449-3ad448366382 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:00.121320 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ce90d-fb9f-4c25-a449-3ad448366382 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:00.121600 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ce90d-fb9f-4c25-a449-3ad448366382 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:00.121873 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ce90d-fb9f-4c25-a449-3ad448366382 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:00.131795 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7ce90d-fb9f-4c25-a449-3ad448366382 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:00.133318 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7ce90d-fb9f-4c25-a449-3ad448366382 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:00.143336 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 933/1853] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:38:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:38:01.078384 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad77659-f654-4689-bd97-495bc038732a None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:01.081153 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7ad77659-f654-4689-bd97-495bc038732a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:01.096174 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad77659-f654-4689-bd97-495bc038732a None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:01.101808 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c10637-b05e-4aef-929a-e5c15778f701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:01.102735 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c10637-b05e-4aef-929a-e5c15778f701 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:01.103182 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c10637-b05e-4aef-929a-e5c15778f701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:01.103598 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c10637-b05e-4aef-929a-e5c15778f701 admin admin] PATH_INFO: `/v3/users/0bd43bca166a4713a4cf76450d02f749` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:01.104955 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c10637-b05e-4aef-929a-e5c15778f701 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bd43bca166a4713a4cf76450d02f749)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:01.107871 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c10637-b05e-4aef-929a-e5c15778f701 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:01.177077 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-75c10637-b05e-4aef-929a-e5c15778f701 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bd43bca166a4713a4cf76450d02f749'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:01.190489 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-75c10637-b05e-4aef-929a-e5c15778f701 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bd43bca166a4713a4cf76450d02f749'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:01.220779 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-75c10637-b05e-4aef-929a-e5c15778f701 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bd43bca166a4713a4cf76450d02f749'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:01.232122 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-75c10637-b05e-4aef-929a-e5c15778f701 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bd43bca166a4713a4cf76450d02f749'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:01.233019 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-75c10637-b05e-4aef-929a-e5c15778f701 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bd43bca166a4713a4cf76450d02f749'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:01.241533 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 921/1854] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:38:01 2022] DELETE /identity/v3/users/0bd43bca166a4713a4cf76450d02f749 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:38:01.246192 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c9e1f6-4fab-4c57-9b3c-ea4799e7718b None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:01.246747 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c9e1f6-4fab-4c57-9b3c-ea4799e7718b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:01.246747 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c9e1f6-4fab-4c57-9b3c-ea4799e7718b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:01.268910 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-83c9e1f6-4fab-4c57-9b3c-ea4799e7718b None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:38:01.270587 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-83c9e1f6-4fab-4c57-9b3c-ea4799e7718b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:01.353237 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 934/1855] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:38:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:38:01.365989 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c10637-b05e-4aef-929a-e5c15778f701 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:01.368137 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-75c10637-b05e-4aef-929a-e5c15778f701 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:01.385834 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:38:01.484961 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c10637-b05e-4aef-929a-e5c15778f701 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:01.492858 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7641f7c5-b731-42b0-87fc-a85cbceb4d9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:01.496068 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7641f7c5-b731-42b0-87fc-a85cbceb4d9e service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:01.496359 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7641f7c5-b731-42b0-87fc-a85cbceb4d9e service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:01.496670 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7641f7c5-b731-42b0-87fc-a85cbceb4d9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:01.523316 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7641f7c5-b731-42b0-87fc-a85cbceb4d9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:01.525029 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7641f7c5-b731-42b0-87fc-a85cbceb4d9e service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:01.551625 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 922/1856] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:38:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:38:02.115812 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c9e1f6-4fab-4c57-9b3c-ea4799e7718b None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:02.117360 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-83c9e1f6-4fab-4c57-9b3c-ea4799e7718b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:02.140724 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c9e1f6-4fab-4c57-9b3c-ea4799e7718b None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:02.157617 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557adb17-ae8c-46d2-b101-ae49d107a922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:02.158444 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557adb17-ae8c-46d2-b101-ae49d107a922 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:02.158824 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557adb17-ae8c-46d2-b101-ae49d107a922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:02.159292 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557adb17-ae8c-46d2-b101-ae49d107a922 admin admin] PATH_INFO: `/v3/users/2fd9e974ca4b47fcad9004a0f474094b` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:02.161053 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557adb17-ae8c-46d2-b101-ae49d107a922 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fd9e974ca4b47fcad9004a0f474094b)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:02.163127 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557adb17-ae8c-46d2-b101-ae49d107a922 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:02.254779 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-557adb17-ae8c-46d2-b101-ae49d107a922 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fd9e974ca4b47fcad9004a0f474094b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:02.264141 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-557adb17-ae8c-46d2-b101-ae49d107a922 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fd9e974ca4b47fcad9004a0f474094b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:02.298820 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-557adb17-ae8c-46d2-b101-ae49d107a922 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fd9e974ca4b47fcad9004a0f474094b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:02.313080 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-557adb17-ae8c-46d2-b101-ae49d107a922 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fd9e974ca4b47fcad9004a0f474094b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:02.313839 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-557adb17-ae8c-46d2-b101-ae49d107a922 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fd9e974ca4b47fcad9004a0f474094b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:02.328125 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 935/1857] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:38:02 2022] DELETE /identity/v3/users/2fd9e974ca4b47fcad9004a0f474094b => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:38:02.333589 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1819c-5d91-476a-bd40-75a9f02d8b29 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:02.334077 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1819c-5d91-476a-bd40-75a9f02d8b29 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:02.334339 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1819c-5d91-476a-bd40-75a9f02d8b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:02.378546 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-63c1819c-5d91-476a-bd40-75a9f02d8b29 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:38:02.380336 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-63c1819c-5d91-476a-bd40-75a9f02d8b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:02.500278 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 923/1858] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:38:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:38:02.540959 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557adb17-ae8c-46d2-b101-ae49d107a922 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:02.542940 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-557adb17-ae8c-46d2-b101-ae49d107a922 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:02.548991 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c1819c-5d91-476a-bd40-75a9f02d8b29 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:02.550533 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-63c1819c-5d91-476a-bd40-75a9f02d8b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:02.580478 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c1819c-5d91-476a-bd40-75a9f02d8b29 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:02.587467 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f874951b-7202-406e-86eb-07d95764820b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:02.588128 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f874951b-7202-406e-86eb-07d95764820b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:02.588408 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f874951b-7202-406e-86eb-07d95764820b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:02.588744 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f874951b-7202-406e-86eb-07d95764820b admin admin] PATH_INFO: `/v3/projects/f50221b5030244819ca4809e31e90404` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:02.593334 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f874951b-7202-406e-86eb-07d95764820b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f50221b5030244819ca4809e31e90404)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:02.597157 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f874951b-7202-406e-86eb-07d95764820b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:02.610571 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:38:02.670133 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557adb17-ae8c-46d2-b101-ae49d107a922 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:02.690297 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99126bbc-055d-4261-9b86-7c7995266dbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:02.691321 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99126bbc-055d-4261-9b86-7c7995266dbe service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:02.691602 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99126bbc-055d-4261-9b86-7c7995266dbe service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:02.691876 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99126bbc-055d-4261-9b86-7c7995266dbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:02.716512 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99126bbc-055d-4261-9b86-7c7995266dbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:02.718083 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99126bbc-055d-4261-9b86-7c7995266dbe service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:02.748935 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 936/1859] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:38:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:38:02.801446 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f874951b-7202-406e-86eb-07d95764820b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f50221b5030244819ca4809e31e90404', 'request_id': 'req-f874951b-7202-406e-86eb-07d95764820b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:02.808838 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f874951b-7202-406e-86eb-07d95764820b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f50221b5030244819ca4809e31e90404', 'request_id': 'req-f874951b-7202-406e-86eb-07d95764820b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:02.816342 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f874951b-7202-406e-86eb-07d95764820b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f50221b5030244819ca4809e31e90404', 'request_id': 'req-f874951b-7202-406e-86eb-07d95764820b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:02.845450 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f874951b-7202-406e-86eb-07d95764820b admin admin] The token cache is being invalidate because project f50221b5030244819ca4809e31e90404 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:38:02.845699 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f874951b-7202-406e-86eb-07d95764820b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:02.846584 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f874951b-7202-406e-86eb-07d95764820b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:02.848204 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 924/1860] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:38:02 2022] DELETE /identity/v3/projects/f50221b5030244819ca4809e31e90404 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:38:02.853408 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc13016-7222-474b-a877-be6d1dc959b8 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:02.853690 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc13016-7222-474b-a877-be6d1dc959b8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:02.853962 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc13016-7222-474b-a877-be6d1dc959b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:02.891224 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-5bc13016-7222-474b-a877-be6d1dc959b8 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:38:02.892441 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5bc13016-7222-474b-a877-be6d1dc959b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:02.985399 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 937/1861] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:38:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:38:02.992276 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7f1cc6-14a4-412d-8906-cd27aa3830f5 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:02.992867 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7f1cc6-14a4-412d-8906-cd27aa3830f5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:02.993244 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7f1cc6-14a4-412d-8906-cd27aa3830f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:03.032858 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-2a7f1cc6-14a4-412d-8906-cd27aa3830f5 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:38:03.034649 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2a7f1cc6-14a4-412d-8906-cd27aa3830f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:03.067475 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 925/1862] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:38:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:38:03.071900 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc13016-7222-474b-a877-be6d1dc959b8 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:03.086319 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc13016-7222-474b-a877-be6d1dc959b8 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:03.097416 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20fdd7e-e97a-4dcb-8bab-3a1179be264b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:03.098744 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20fdd7e-e97a-4dcb-8bab-3a1179be264b admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:03.098744 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20fdd7e-e97a-4dcb-8bab-3a1179be264b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:03.098744 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20fdd7e-e97a-4dcb-8bab-3a1179be264b admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:03.099329 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20fdd7e-e97a-4dcb-8bab-3a1179be264b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:38:03.099632 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20fdd7e-e97a-4dcb-8bab-3a1179be264b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:03.106225 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20fdd7e-e97a-4dcb-8bab-3a1179be264b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:03.134167 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 938/1863] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:38:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:38:03.138313 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7f1cc6-14a4-412d-8906-cd27aa3830f5 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:03.160418 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7f1cc6-14a4-412d-8906-cd27aa3830f5 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:03.176878 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473d4f3f-61b7-42d4-b667-7c5c0bef95e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:03.194740 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d4f3f-61b7-42d4-b667-7c5c0bef95e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:03.196079 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d4f3f-61b7-42d4-b667-7c5c0bef95e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:03.196501 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d4f3f-61b7-42d4-b667-7c5c0bef95e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:03.198063 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473d4f3f-61b7-42d4-b667-7c5c0bef95e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:03.200748 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473d4f3f-61b7-42d4-b667-7c5c0bef95e5 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:03.259397 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 926/1864] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:38:03 2022] POST /identity/v3/projects => generated 381 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:38:03.261057 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20fdd7e-e97a-4dcb-8bab-3a1179be264b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:03.272917 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20fdd7e-e97a-4dcb-8bab-3a1179be264b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:03.376713 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00975f3b-9734-4a30-b2bf-6e4e11601bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:03.377471 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00975f3b-9734-4a30-b2bf-6e4e11601bc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:03.377856 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00975f3b-9734-4a30-b2bf-6e4e11601bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:03.378258 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00975f3b-9734-4a30-b2bf-6e4e11601bc6 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:03.379750 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00975f3b-9734-4a30-b2bf-6e4e11601bc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:03.400620 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00975f3b-9734-4a30-b2bf-6e4e11601bc6 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:03.442974 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 939/1865] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:38:03 2022] POST /identity/v3/users => generated 291 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:38:03.446960 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473d4f3f-61b7-42d4-b667-7c5c0bef95e5 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:03.460486 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473d4f3f-61b7-42d4-b667-7c5c0bef95e5 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:03.470288 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75fb059-af81-4fef-b320-667f5e6a6c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:03.471174 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75fb059-af81-4fef-b320-667f5e6a6c4a admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:03.471816 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75fb059-af81-4fef-b320-667f5e6a6c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:03.473131 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75fb059-af81-4fef-b320-667f5e6a6c4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:03.475030 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75fb059-af81-4fef-b320-667f5e6a6c4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:03.476704 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75fb059-af81-4fef-b320-667f5e6a6c4a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:03.483651 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:38:03.494377 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 927/1866] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:38:03 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:38:03.497932 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00975f3b-9734-4a30-b2bf-6e4e11601bc6 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:03.509514 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00975f3b-9734-4a30-b2bf-6e4e11601bc6 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:03.517885 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb2a66-5280-4927-a0f1-685894d90b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:03.518705 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb2a66-5280-4927-a0f1-685894d90b05 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:03.519448 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb2a66-5280-4927-a0f1-685894d90b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:03.519829 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb2a66-5280-4927-a0f1-685894d90b05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:03.520863 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bb2a66-5280-4927-a0f1-685894d90b05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:03.523361 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bb2a66-5280-4927-a0f1-685894d90b05 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:03.543235 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 940/1867] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:38:03 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 09:38:03.547356 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75fb059-af81-4fef-b320-667f5e6a6c4a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:03.562801 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75fb059-af81-4fef-b320-667f5e6a6c4a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:03.570276 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d4f82-7e77-4407-9c2d-21358eb8743b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:03.571325 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d4f82-7e77-4407-9c2d-21358eb8743b admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:03.571855 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d4f82-7e77-4407-9c2d-21358eb8743b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:03.573480 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d4f82-7e77-4407-9c2d-21358eb8743b admin admin] PATH_INFO: `/v3/projects/9dc4fbf213f04cd7ab49abad693a1cf2/users/b4e4e3b5ce544725b1f4a439596314a8/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:03.603688 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17d4f82-7e77-4407-9c2d-21358eb8743b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dc4fbf213f04cd7ab49abad693a1cf2, user_id=b4e4e3b5ce544725b1f4a439596314a8, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:03.608868 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17d4f82-7e77-4407-9c2d-21358eb8743b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:03.640278 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 928/1868] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:38:03 2022] PUT /identity/v3/projects/9dc4fbf213f04cd7ab49abad693a1cf2/users/b4e4e3b5ce544725b1f4a439596314a8/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:38:03.647511 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a4c346-888c-491f-aee1-829ed1597277 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:03.647511 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a4c346-888c-491f-aee1-829ed1597277 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:03.649752 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a4c346-888c-491f-aee1-829ed1597277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:03.681444 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-c5a4c346-888c-491f-aee1-829ed1597277 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:38:03.684415 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c5a4c346-888c-491f-aee1-829ed1597277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:03.710605 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d4f82-7e77-4407-9c2d-21358eb8743b admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:03.717062 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d17d4f82-7e77-4407-9c2d-21358eb8743b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:03.818273 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:38:03.980925 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 941/1869] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:38:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:38:04.000054 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d4f82-7e77-4407-9c2d-21358eb8743b admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:04.005315 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ff992-40a7-4b65-b359-ef0723141a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:04.006433 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ff992-40a7-4b65-b359-ef0723141a87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:04.006858 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ff992-40a7-4b65-b359-ef0723141a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:04.007220 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ff992-40a7-4b65-b359-ef0723141a87 admin admin] PATH_INFO: `/v3/projects/c72f89f60e2245e2aa5c287ca187cfa2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:04.009093 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72ff992-40a7-4b65-b359-ef0723141a87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c72f89f60e2245e2aa5c287ca187cfa2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:04.011246 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72ff992-40a7-4b65-b359-ef0723141a87 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:04.098434 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a4c346-888c-491f-aee1-829ed1597277 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:04.099966 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c5a4c346-888c-491f-aee1-829ed1597277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:04.119013 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d72ff992-40a7-4b65-b359-ef0723141a87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c72f89f60e2245e2aa5c287ca187cfa2', 'request_id': 'req-d72ff992-40a7-4b65-b359-ef0723141a87'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:04.137683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d72ff992-40a7-4b65-b359-ef0723141a87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c72f89f60e2245e2aa5c287ca187cfa2', 'request_id': 'req-d72ff992-40a7-4b65-b359-ef0723141a87'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:04.151034 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d72ff992-40a7-4b65-b359-ef0723141a87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c72f89f60e2245e2aa5c287ca187cfa2', 'request_id': 'req-d72ff992-40a7-4b65-b359-ef0723141a87'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:04.179789 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d72ff992-40a7-4b65-b359-ef0723141a87 admin admin] The token cache is being invalidate because project c72f89f60e2245e2aa5c287ca187cfa2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:38:04.180194 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d72ff992-40a7-4b65-b359-ef0723141a87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:04.182531 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d72ff992-40a7-4b65-b359-ef0723141a87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:04.185559 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 929/1870] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:38:03 2022] DELETE /identity/v3/projects/c72f89f60e2245e2aa5c287ca187cfa2 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:38:04.191163 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61facbba-673d-4746-a305-7a938bcaa0c0 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:04.191538 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61facbba-673d-4746-a305-7a938bcaa0c0 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:04.191851 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61facbba-673d-4746-a305-7a938bcaa0c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:04.195497 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a4c346-888c-491f-aee1-829ed1597277 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:04.241674 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-61facbba-673d-4746-a305-7a938bcaa0c0 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:38:04.242924 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-61facbba-673d-4746-a305-7a938bcaa0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:04.264481 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a8d365-b942-4d0e-8a5c-8b96908df6a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:04.265739 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a8d365-b942-4d0e-8a5c-8b96908df6a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:04.266198 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a8d365-b942-4d0e-8a5c-8b96908df6a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:04.266676 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a8d365-b942-4d0e-8a5c-8b96908df6a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:04.274814 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-42a8d365-b942-4d0e-8a5c-8b96908df6a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:04.362375 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 930/1871] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:38:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:38:04.370261 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4276fcb-b1a9-485b-8858-82c5711f52b7 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:04.371016 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4276fcb-b1a9-485b-8858-82c5711f52b7 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:04.371016 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4276fcb-b1a9-485b-8858-82c5711f52b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:04.387088 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:38:04.410272 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a8d365-b942-4d0e-8a5c-8b96908df6a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:04.410913 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-e4276fcb-b1a9-485b-8858-82c5711f52b7 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:38:04.411759 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a8d365-b942-4d0e-8a5c-8b96908df6a3 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:04.413654 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e4276fcb-b1a9-485b-8858-82c5711f52b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:04.425396 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 931/1872] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:38:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:38:04.429973 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4276fcb-b1a9-485b-8858-82c5711f52b7 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:04.434615 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 942/1873] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:38:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 09:38:04.460851 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4276fcb-b1a9-485b-8858-82c5711f52b7 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:04.467190 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cddf9ef-6751-4b92-a434-1e205d9e466c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:04.467821 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cddf9ef-6751-4b92-a434-1e205d9e466c admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:04.468100 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cddf9ef-6751-4b92-a434-1e205d9e466c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:04.468372 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cddf9ef-6751-4b92-a434-1e205d9e466c admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:04.469072 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cddf9ef-6751-4b92-a434-1e205d9e466c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:38:04.469376 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cddf9ef-6751-4b92-a434-1e205d9e466c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:04.471021 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cddf9ef-6751-4b92-a434-1e205d9e466c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:04.488775 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 932/1874] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:38:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:38:04.489680 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a8d365-b942-4d0e-8a5c-8b96908df6a3 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:04.500684 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a8d365-b942-4d0e-8a5c-8b96908df6a3 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:04.507130 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b07c7-2ab5-452b-af1d-e72da4176a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:04.508173 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b07c7-2ab5-452b-af1d-e72da4176a7d admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:04.508316 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b07c7-2ab5-452b-af1d-e72da4176a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:04.508778 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b07c7-2ab5-452b-af1d-e72da4176a7d admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:04.509788 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b07c7-2ab5-452b-af1d-e72da4176a7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:04.513264 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b07c7-2ab5-452b-af1d-e72da4176a7d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:04.550479 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 943/1875] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:38:04 2022] POST /identity/v3/projects => generated 367 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:38:04.552039 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cddf9ef-6751-4b92-a434-1e205d9e466c admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:04.565753 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cddf9ef-6751-4b92-a434-1e205d9e466c admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:04.639808 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3176e72e-96c9-42d8-bcb1-8f6398feb90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:04.640704 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176e72e-96c9-42d8-bcb1-8f6398feb90e admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:04.641177 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176e72e-96c9-42d8-bcb1-8f6398feb90e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:04.641705 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176e72e-96c9-42d8-bcb1-8f6398feb90e admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:04.642285 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3176e72e-96c9-42d8-bcb1-8f6398feb90e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:04.645520 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3176e72e-96c9-42d8-bcb1-8f6398feb90e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:04.680848 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 933/1876] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:38:04 2022] POST /identity/v3/users => generated 284 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:38:04.682079 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b07c7-2ab5-452b-af1d-e72da4176a7d admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:04.693799 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b07c7-2ab5-452b-af1d-e72da4176a7d admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:04.698602 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3590f914-d29f-4507-8694-4a2bf7b407a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:04.699762 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590f914-d29f-4507-8694-4a2bf7b407a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:04.701068 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590f914-d29f-4507-8694-4a2bf7b407a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:04.701068 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590f914-d29f-4507-8694-4a2bf7b407a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:04.701284 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3590f914-d29f-4507-8694-4a2bf7b407a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:04.703790 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3590f914-d29f-4507-8694-4a2bf7b407a6 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:04.720597 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 944/1877] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:38:04 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 09:38:04.723895 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3176e72e-96c9-42d8-bcb1-8f6398feb90e admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:04.735446 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3176e72e-96c9-42d8-bcb1-8f6398feb90e admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:04.740873 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c11cfb-991c-4b83-b551-57a2c6a9e4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:04.742129 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c11cfb-991c-4b83-b551-57a2c6a9e4b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:04.742129 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c11cfb-991c-4b83-b551-57a2c6a9e4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:04.742129 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c11cfb-991c-4b83-b551-57a2c6a9e4b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:04.743048 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c11cfb-991c-4b83-b551-57a2c6a9e4b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:04.745280 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c11cfb-991c-4b83-b551-57a2c6a9e4b8 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:04.758861 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 934/1878] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:38:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:38:04.765823 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3590f914-d29f-4507-8694-4a2bf7b407a6 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:04.778103 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3590f914-d29f-4507-8694-4a2bf7b407a6 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:04.784223 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0cf8f-c688-4d0b-8945-b0be2610f3fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:04.785533 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0cf8f-c688-4d0b-8945-b0be2610f3fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:04.785533 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0cf8f-c688-4d0b-8945-b0be2610f3fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:04.785758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0cf8f-c688-4d0b-8945-b0be2610f3fd admin admin] PATH_INFO: `/v3/projects/2c69f633a97d49caa0ae0123eacacd4c/users/0c33508f209b40628451de85db4e71e5/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:04.790068 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:38:04.810128 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb0cf8f-c688-4d0b-8945-b0be2610f3fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c69f633a97d49caa0ae0123eacacd4c, user_id=0c33508f209b40628451de85db4e71e5, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:04.813189 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb0cf8f-c688-4d0b-8945-b0be2610f3fd admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:04.841935 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 945/1879] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:38:04 2022] PUT /identity/v3/projects/2c69f633a97d49caa0ae0123eacacd4c/users/0c33508f209b40628451de85db4e71e5/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:38:04.845280 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a8c1a3-7323-4fd7-84ca-3c7c74452c15 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:04.846018 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a8c1a3-7323-4fd7-84ca-3c7c74452c15 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:04.846360 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a8c1a3-7323-4fd7-84ca-3c7c74452c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:04.879975 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a1a8c1a3-7323-4fd7-84ca-3c7c74452c15 None None] MFA Rules not processed for user `0c33508f209b40628451de85db4e71e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:38:04.881325 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a1a8c1a3-7323-4fd7-84ca-3c7c74452c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:04.968808 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 935/1880] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:38:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:38:04.977770 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0cf8f-c688-4d0b-8945-b0be2610f3fd admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:04.979697 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4bb0cf8f-c688-4d0b-8945-b0be2610f3fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:05.093358 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0cf8f-c688-4d0b-8945-b0be2610f3fd admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:05.098115 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e429c8-0172-4ede-a85a-a6cb523f9c39 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72b0538f0e42446887b1d95b2fb41fb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:05.098805 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e429c8-0172-4ede-a85a-a6cb523f9c39 service glance] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:05.099141 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e429c8-0172-4ede-a85a-a6cb523f9c39 service glance] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:05.099437 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e429c8-0172-4ede-a85a-a6cb523f9c39 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:05.125795 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e429c8-0172-4ede-a85a-a6cb523f9c39 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:05.128286 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e429c8-0172-4ede-a85a-a6cb523f9c39 service glance] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:05.150635 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 946/1881] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:38:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 09:38:06.643680 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f586e9-c0e4-485b-a1f5-9c7b568e9686 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:06.644159 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f586e9-c0e4-485b-a1f5-9c7b568e9686 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:06.644248 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f586e9-c0e4-485b-a1f5-9c7b568e9686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:06.687186 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-32f586e9-c0e4-485b-a1f5-9c7b568e9686 None None] MFA Rules not processed for user `f7d2c6275669404db608d714baeb0065`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:38:06.688661 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-32f586e9-c0e4-485b-a1f5-9c7b568e9686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:06.742088 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:38:06.787102 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 936/1882] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:38:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:38:06.808879 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e429c8-0172-4ede-a85a-a6cb523f9c39 service glance] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:06.812194 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-23e429c8-0172-4ede-a85a-a6cb523f9c39 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:06.845224 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c744ec5a-1bb3-40c4-925e-548849c3070d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:06.845224 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c744ec5a-1bb3-40c4-925e-548849c3070d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:06.845224 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c744ec5a-1bb3-40c4-925e-548849c3070d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:06.897849 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-c744ec5a-1bb3-40c4-925e-548849c3070d None None] MFA Rules not processed for user `b4e4e3b5ce544725b1f4a439596314a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:38:06.899511 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c744ec5a-1bb3-40c4-925e-548849c3070d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:06.995100 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e429c8-0172-4ede-a85a-a6cb523f9c39 service glance] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:07.004283 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d2b083-3610-449f-ac73-71e5d2c282bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe4a6e4b6184826b30ac4fc13ad9b1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:07.005456 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d2b083-3610-449f-ac73-71e5d2c282bd service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:07.005933 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d2b083-3610-449f-ac73-71e5d2c282bd service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:07.006385 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d2b083-3610-449f-ac73-71e5d2c282bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:07.018138 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 937/1883] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:38:06 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:38:07.022816 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d2b083-3610-449f-ac73-71e5d2c282bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:07.024441 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d2b083-3610-449f-ac73-71e5d2c282bd service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:07.031341 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c744ec5a-1bb3-40c4-925e-548849c3070d None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:07.032786 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c744ec5a-1bb3-40c4-925e-548849c3070d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:07.050106 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 947/1884] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:38:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:38:07.181392 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c744ec5a-1bb3-40c4-925e-548849c3070d None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:07.193279 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd294d19-72d4-4b08-8c89-efedefc3477a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:07.194181 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd294d19-72d4-4b08-8c89-efedefc3477a service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:07.194659 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd294d19-72d4-4b08-8c89-efedefc3477a service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:07.195070 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd294d19-72d4-4b08-8c89-efedefc3477a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:07.215307 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd294d19-72d4-4b08-8c89-efedefc3477a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:07.218003 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd294d19-72d4-4b08-8c89-efedefc3477a service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:07.228940 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 938/1885] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:38:07 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:38:08.900926 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3204043c-453b-4ddd-9427-e8b30c080942 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:08.900926 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3204043c-453b-4ddd-9427-e8b30c080942 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:08.901576 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3204043c-453b-4ddd-9427-e8b30c080942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:08.929475 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-3204043c-453b-4ddd-9427-e8b30c080942 None None] MFA Rules not processed for user `f7d2c6275669404db608d714baeb0065`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:38:08.930887 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3204043c-453b-4ddd-9427-e8b30c080942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:08.945819 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 948/1886] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:38:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:38:08.949370 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1886db49-2703-4598-8f8f-e90134bb260b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:08.949812 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1886db49-2703-4598-8f8f-e90134bb260b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:08.949891 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1886db49-2703-4598-8f8f-e90134bb260b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:08.970270 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-1886db49-2703-4598-8f8f-e90134bb260b None None] MFA Rules not processed for user `b4e4e3b5ce544725b1f4a439596314a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:38:08.971485 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1886db49-2703-4598-8f8f-e90134bb260b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:08.979552 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 939/1887] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:38:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:38:09.032617 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3204043c-453b-4ddd-9427-e8b30c080942 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:09.034562 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3204043c-453b-4ddd-9427-e8b30c080942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:09.123678 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:38:09.174040 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3204043c-453b-4ddd-9427-e8b30c080942 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:09.186946 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0148f2b3-8cd5-4c39-92e5-4acf1b7bfc47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:09.187681 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0148f2b3-8cd5-4c39-92e5-4acf1b7bfc47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:09.191933 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0148f2b3-8cd5-4c39-92e5-4acf1b7bfc47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:09.191933 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0148f2b3-8cd5-4c39-92e5-4acf1b7bfc47 admin admin] PATH_INFO: `/v3/users/0bf5747312844825bfaabc33e6228593` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:09.191933 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0148f2b3-8cd5-4c39-92e5-4acf1b7bfc47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bf5747312844825bfaabc33e6228593)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:09.201877 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0148f2b3-8cd5-4c39-92e5-4acf1b7bfc47 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:09.308687 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0148f2b3-8cd5-4c39-92e5-4acf1b7bfc47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bf5747312844825bfaabc33e6228593'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:09.316990 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0148f2b3-8cd5-4c39-92e5-4acf1b7bfc47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bf5747312844825bfaabc33e6228593'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:09.330865 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0148f2b3-8cd5-4c39-92e5-4acf1b7bfc47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bf5747312844825bfaabc33e6228593'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:09.339071 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0148f2b3-8cd5-4c39-92e5-4acf1b7bfc47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bf5747312844825bfaabc33e6228593'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:09.340193 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0148f2b3-8cd5-4c39-92e5-4acf1b7bfc47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bf5747312844825bfaabc33e6228593'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:09.346688 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 949/1888] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:38:09 2022] DELETE /identity/v3/users/0bf5747312844825bfaabc33e6228593 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:38:09.351375 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b677296-b3f1-43dd-96fa-d3ad2cd46464 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:09.351799 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b677296-b3f1-43dd-96fa-d3ad2cd46464 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:09.352112 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b677296-b3f1-43dd-96fa-d3ad2cd46464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:09.375333 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-6b677296-b3f1-43dd-96fa-d3ad2cd46464 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:38:09.377033 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6b677296-b3f1-43dd-96fa-d3ad2cd46464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:09.455824 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 940/1889] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:38:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:38:09.467056 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0148f2b3-8cd5-4c39-92e5-4acf1b7bfc47 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:09.468889 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0148f2b3-8cd5-4c39-92e5-4acf1b7bfc47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:09.547132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0148f2b3-8cd5-4c39-92e5-4acf1b7bfc47 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:09.555416 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a44557-f2e8-454b-a680-db89cedf1c17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:09.556048 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a44557-f2e8-454b-a680-db89cedf1c17 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:09.556323 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a44557-f2e8-454b-a680-db89cedf1c17 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:09.556628 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a44557-f2e8-454b-a680-db89cedf1c17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:09.577424 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a44557-f2e8-454b-a680-db89cedf1c17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:09.578887 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a44557-f2e8-454b-a680-db89cedf1c17 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:09.589940 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 950/1890] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:38:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:38:09.929486 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b677296-b3f1-43dd-96fa-d3ad2cd46464 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:09.930999 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6b677296-b3f1-43dd-96fa-d3ad2cd46464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:09.951116 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b677296-b3f1-43dd-96fa-d3ad2cd46464 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:09.957750 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d50844-9609-46ff-bd07-ea28a1c14158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:09.958753 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d50844-9609-46ff-bd07-ea28a1c14158 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:09.959271 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d50844-9609-46ff-bd07-ea28a1c14158 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:09.959799 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d50844-9609-46ff-bd07-ea28a1c14158 admin admin] PATH_INFO: `/v3/projects/3aa3b2710c6548fbabaf2a2095fe0431` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:09.961385 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d50844-9609-46ff-bd07-ea28a1c14158 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3aa3b2710c6548fbabaf2a2095fe0431)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:09.963517 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d50844-9609-46ff-bd07-ea28a1c14158 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:09.967516 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:38:10.043451 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e3d50844-9609-46ff-bd07-ea28a1c14158 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3aa3b2710c6548fbabaf2a2095fe0431', 'request_id': 'req-e3d50844-9609-46ff-bd07-ea28a1c14158'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:10.052727 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e3d50844-9609-46ff-bd07-ea28a1c14158 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3aa3b2710c6548fbabaf2a2095fe0431', 'request_id': 'req-e3d50844-9609-46ff-bd07-ea28a1c14158'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:10.061506 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e3d50844-9609-46ff-bd07-ea28a1c14158 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3aa3b2710c6548fbabaf2a2095fe0431', 'request_id': 'req-e3d50844-9609-46ff-bd07-ea28a1c14158'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:10.080326 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e3d50844-9609-46ff-bd07-ea28a1c14158 admin admin] The token cache is being invalidate because project 3aa3b2710c6548fbabaf2a2095fe0431 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:38:10.080658 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e3d50844-9609-46ff-bd07-ea28a1c14158 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:10.081765 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e3d50844-9609-46ff-bd07-ea28a1c14158 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:10.083859 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 941/1891] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:38:09 2022] DELETE /identity/v3/projects/3aa3b2710c6548fbabaf2a2095fe0431 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:38:10.088954 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251731ed-448a-49ff-9baa-78f36acda246 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:10.089966 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251731ed-448a-49ff-9baa-78f36acda246 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:10.090079 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251731ed-448a-49ff-9baa-78f36acda246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:10.114764 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-251731ed-448a-49ff-9baa-78f36acda246 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:38:10.115457 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-251731ed-448a-49ff-9baa-78f36acda246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:10.216474 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 951/1892] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:38:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:38:10.225114 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266c81f-7ff4-4fb8-9e21-c7ff481f17e2 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:10.226001 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266c81f-7ff4-4fb8-9e21-c7ff481f17e2 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:10.226806 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266c81f-7ff4-4fb8-9e21-c7ff481f17e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:10.278365 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b266c81f-7ff4-4fb8-9e21-c7ff481f17e2 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:38:10.295205 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b266c81f-7ff4-4fb8-9e21-c7ff481f17e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:10.338972 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 942/1893] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:38:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:38:10.343978 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251731ed-448a-49ff-9baa-78f36acda246 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:10.354656 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251731ed-448a-49ff-9baa-78f36acda246 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:10.381810 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95023ab-8434-483c-afe2-b59f0f7fd2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:10.383987 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95023ab-8434-483c-afe2-b59f0f7fd2de admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:10.384861 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95023ab-8434-483c-afe2-b59f0f7fd2de admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:10.388080 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95023ab-8434-483c-afe2-b59f0f7fd2de admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:10.389204 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95023ab-8434-483c-afe2-b59f0f7fd2de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:38:10.389747 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95023ab-8434-483c-afe2-b59f0f7fd2de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:10.403869 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95023ab-8434-483c-afe2-b59f0f7fd2de admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:10.410380 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:38:10.446004 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 952/1894] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:38:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:38:10.451995 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b266c81f-7ff4-4fb8-9e21-c7ff481f17e2 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:10.464019 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b266c81f-7ff4-4fb8-9e21-c7ff481f17e2 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:10.472470 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4049ea17-2173-48aa-a94c-4d2423756183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:10.473100 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4049ea17-2173-48aa-a94c-4d2423756183 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:10.473380 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4049ea17-2173-48aa-a94c-4d2423756183 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:10.473652 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4049ea17-2173-48aa-a94c-4d2423756183 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:10.474312 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4049ea17-2173-48aa-a94c-4d2423756183 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:10.477795 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4049ea17-2173-48aa-a94c-4d2423756183 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:10.522382 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 943/1895] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:38:10 2022] POST /identity/v3/projects => generated 373 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:38:10.526609 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95023ab-8434-483c-afe2-b59f0f7fd2de admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:10.542761 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95023ab-8434-483c-afe2-b59f0f7fd2de admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:10.665722 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d42ff45-005a-4e20-b686-bc9945dc416b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:10.666662 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d42ff45-005a-4e20-b686-bc9945dc416b admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:10.667282 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d42ff45-005a-4e20-b686-bc9945dc416b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:10.667788 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d42ff45-005a-4e20-b686-bc9945dc416b admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:10.668653 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d42ff45-005a-4e20-b686-bc9945dc416b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:10.671162 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d42ff45-005a-4e20-b686-bc9945dc416b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:10.717741 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 953/1896] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:38:10 2022] POST /identity/v3/users => generated 287 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:38:10.723155 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4049ea17-2173-48aa-a94c-4d2423756183 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:10.737080 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4049ea17-2173-48aa-a94c-4d2423756183 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:10.749283 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774fe678-78cf-4c7b-b805-f2d751bfdc80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:10.750202 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fe678-78cf-4c7b-b805-f2d751bfdc80 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:10.750202 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fe678-78cf-4c7b-b805-f2d751bfdc80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:10.750436 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fe678-78cf-4c7b-b805-f2d751bfdc80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:10.751148 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774fe678-78cf-4c7b-b805-f2d751bfdc80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:10.758674 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774fe678-78cf-4c7b-b805-f2d751bfdc80 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:10.783081 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 944/1897] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:38:10 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:38:10.788568 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d42ff45-005a-4e20-b686-bc9945dc416b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:10.847492 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d42ff45-005a-4e20-b686-bc9945dc416b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:10.857298 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b64c70-4ebc-4b99-86c8-fc5b1af7384d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:10.857931 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b64c70-4ebc-4b99-86c8-fc5b1af7384d admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:10.858203 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b64c70-4ebc-4b99-86c8-fc5b1af7384d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:10.858478 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b64c70-4ebc-4b99-86c8-fc5b1af7384d admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:10.859339 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b64c70-4ebc-4b99-86c8-fc5b1af7384d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:10.861347 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b64c70-4ebc-4b99-86c8-fc5b1af7384d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:10.874241 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 954/1898] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:38:10 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:38:10.878209 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774fe678-78cf-4c7b-b805-f2d751bfdc80 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:10.894064 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774fe678-78cf-4c7b-b805-f2d751bfdc80 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:10.900753 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156852eb-eb67-4421-aae0-fbcadd9bfe84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:10.901714 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156852eb-eb67-4421-aae0-fbcadd9bfe84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:10.902206 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156852eb-eb67-4421-aae0-fbcadd9bfe84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:10.902661 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156852eb-eb67-4421-aae0-fbcadd9bfe84 admin admin] PATH_INFO: `/v3/projects/f7b8fb656ee34fae8d20679c0e0f7ff9/users/0968b48cb99d4168b36a2602bd87f788/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:10.940752 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156852eb-eb67-4421-aae0-fbcadd9bfe84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7b8fb656ee34fae8d20679c0e0f7ff9, user_id=0968b48cb99d4168b36a2602bd87f788, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:10.940882 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156852eb-eb67-4421-aae0-fbcadd9bfe84 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:10.984374 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 945/1899] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:38:10 2022] PUT /identity/v3/projects/f7b8fb656ee34fae8d20679c0e0f7ff9/users/0968b48cb99d4168b36a2602bd87f788/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:38:11.001409 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee79c17-7a49-45f8-9734-fd90b421b8d5 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:11.002020 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee79c17-7a49-45f8-9734-fd90b421b8d5 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:11.002496 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee79c17-7a49-45f8-9734-fd90b421b8d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:11.033389 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-0ee79c17-7a49-45f8-9734-fd90b421b8d5 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:38:11.035374 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0ee79c17-7a49-45f8-9734-fd90b421b8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:11.116544 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 955/1900] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:38:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:38:11.128008 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156852eb-eb67-4421-aae0-fbcadd9bfe84 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:11.129856 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-156852eb-eb67-4421-aae0-fbcadd9bfe84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:11.206393 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156852eb-eb67-4421-aae0-fbcadd9bfe84 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:11.212652 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5470ca29-df23-418f-85dc-13a5a04eaf6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:11.214450 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470ca29-df23-418f-85dc-13a5a04eaf6a service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:11.214941 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470ca29-df23-418f-85dc-13a5a04eaf6a service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:11.215543 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470ca29-df23-418f-85dc-13a5a04eaf6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:11.235446 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5470ca29-df23-418f-85dc-13a5a04eaf6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:11.237163 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5470ca29-df23-418f-85dc-13a5a04eaf6a service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:11.249855 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 946/1901] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:38:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:38:18.660651 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278117a4-229c-4640-a42b-ae90ff74deef None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:18.661561 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278117a4-229c-4640-a42b-ae90ff74deef None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:18.661981 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278117a4-229c-4640-a42b-ae90ff74deef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:18.709826 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-278117a4-229c-4640-a42b-ae90ff74deef None None] MFA Rules not processed for user `0968b48cb99d4168b36a2602bd87f788`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:38:18.711311 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-278117a4-229c-4640-a42b-ae90ff74deef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:18.850907 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5470ca29-df23-418f-85dc-13a5a04eaf6a service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:18.851442 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 956/1902] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:38:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:38:18.864834 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5470ca29-df23-418f-85dc-13a5a04eaf6a service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:18.869891 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ccb58-202f-47c7-b1ec-a1c1aa13b6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:18.870770 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ccb58-202f-47c7-b1ec-a1c1aa13b6df admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:18.871229 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ccb58-202f-47c7-b1ec-a1c1aa13b6df admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:18.871689 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ccb58-202f-47c7-b1ec-a1c1aa13b6df admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:18.872498 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096ccb58-202f-47c7-b1ec-a1c1aa13b6df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:18.874628 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096ccb58-202f-47c7-b1ec-a1c1aa13b6df admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:18.908992 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 947/1903] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:38:18 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:38:18.909807 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278117a4-229c-4640-a42b-ae90ff74deef None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:18.915807 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:38:18.925948 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278117a4-229c-4640-a42b-ae90ff74deef None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:18.942000 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5167f4b-3e33-41e6-8017-1d2eecaf47fb None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:18.942618 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5167f4b-3e33-41e6-8017-1d2eecaf47fb None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:18.943088 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5167f4b-3e33-41e6-8017-1d2eecaf47fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:18.970712 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-d5167f4b-3e33-41e6-8017-1d2eecaf47fb None None] MFA Rules not processed for user `7f389510a6a044d8a7487561b169544b`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:38:18.972019 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d5167f4b-3e33-41e6-8017-1d2eecaf47fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:18.977536 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:38:19.017838 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea7b3b0-7338-4989-959d-38d3240e4e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:19.019424 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7b3b0-7338-4989-959d-38d3240e4e8a admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:19.019727 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7b3b0-7338-4989-959d-38d3240e4e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:19.020003 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7b3b0-7338-4989-959d-38d3240e4e8a admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:19.020944 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea7b3b0-7338-4989-959d-38d3240e4e8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:19.023306 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea7b3b0-7338-4989-959d-38d3240e4e8a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:19.055109 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 957/1904] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:38:18 2022] POST /identity/v3/users => generated 292 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:38:19.060701 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea7b3b0-7338-4989-959d-38d3240e4e8a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:19.068988 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 948/1905] 10.222.0.70 () {58 vars in 1009 bytes} [Thu Oct 27 09:38:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:38:19.071569 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea7b3b0-7338-4989-959d-38d3240e4e8a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:19.081421 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39de3ba-4369-40ec-bc5a-ab1f20646568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:19.082255 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39de3ba-4369-40ec-bc5a-ab1f20646568 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:19.082695 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39de3ba-4369-40ec-bc5a-ab1f20646568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:19.083112 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39de3ba-4369-40ec-bc5a-ab1f20646568 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:19.083898 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39de3ba-4369-40ec-bc5a-ab1f20646568 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:19.085710 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39de3ba-4369-40ec-bc5a-ab1f20646568 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:19.098551 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 958/1906] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:38:19 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 09:38:19.102435 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5167f4b-3e33-41e6-8017-1d2eecaf47fb None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:19.126481 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39de3ba-4369-40ec-bc5a-ab1f20646568 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:19.127837 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5167f4b-3e33-41e6-8017-1d2eecaf47fb None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:19.128170 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c39de3ba-4369-40ec-bc5a-ab1f20646568 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:19.139612 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585b786a-fac8-4584-b5b9-337d92e7e29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:19.140154 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585b786a-fac8-4584-b5b9-337d92e7e29e admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:19.140428 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585b786a-fac8-4584-b5b9-337d92e7e29e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:19.140720 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585b786a-fac8-4584-b5b9-337d92e7e29e admin admin] PATH_INFO: `/v3/projects/d6bd77f2911f4d7fb339ce7d80cf4094/users/655a2d89f6e440bc8634c5d8aab17ccf/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:19.163879 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585b786a-fac8-4584-b5b9-337d92e7e29e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6bd77f2911f4d7fb339ce7d80cf4094, user_id=655a2d89f6e440bc8634c5d8aab17ccf, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:19.168406 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585b786a-fac8-4584-b5b9-337d92e7e29e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:19.194989 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 949/1907] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:38:19 2022] PUT /identity/v3/projects/d6bd77f2911f4d7fb339ce7d80cf4094/users/655a2d89f6e440bc8634c5d8aab17ccf/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:38:19.198871 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585b786a-fac8-4584-b5b9-337d92e7e29e admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:19.232359 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39de3ba-4369-40ec-bc5a-ab1f20646568 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:19.280323 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585b786a-fac8-4584-b5b9-337d92e7e29e admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:19.304898 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c91b4-5ecc-4e5d-a350-2733fdf885c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:19.306130 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c91b4-5ecc-4e5d-a350-2733fdf885c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:19.307037 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c91b4-5ecc-4e5d-a350-2733fdf885c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:19.307803 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c91b4-5ecc-4e5d-a350-2733fdf885c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:19.308457 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c91b4-5ecc-4e5d-a350-2733fdf885c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:19.310104 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c91b4-5ecc-4e5d-a350-2733fdf885c1 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:19.324119 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 950/1908] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:38:19 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:38:19.328124 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c91b4-5ecc-4e5d-a350-2733fdf885c1 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:19.334378 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e317a92e-d386-40a2-9252-3ab17a883a5c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eecf79ea4700496c89b1a67c614b34d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:19.335135 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e317a92e-d386-40a2-9252-3ab17a883a5c service swift] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:19.335677 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e317a92e-d386-40a2-9252-3ab17a883a5c service swift] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:19.336108 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e317a92e-d386-40a2-9252-3ab17a883a5c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:19.341683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c91b4-5ecc-4e5d-a350-2733fdf885c1 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:19.346672 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d16ec7-8607-4bea-9345-b57ce8a2e613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:19.347344 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d16ec7-8607-4bea-9345-b57ce8a2e613 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:19.347618 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d16ec7-8607-4bea-9345-b57ce8a2e613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:19.347889 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d16ec7-8607-4bea-9345-b57ce8a2e613 admin admin] PATH_INFO: `/v3/projects/d6bd77f2911f4d7fb339ce7d80cf4094/users/655a2d89f6e440bc8634c5d8aab17ccf/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:19.349909 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d16ec7-8607-4bea-9345-b57ce8a2e613 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6bd77f2911f4d7fb339ce7d80cf4094, user_id=655a2d89f6e440bc8634c5d8aab17ccf, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:19.352707 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d16ec7-8607-4bea-9345-b57ce8a2e613 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:19.374018 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 951/1909] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:38:19 2022] PUT /identity/v3/projects/d6bd77f2911f4d7fb339ce7d80cf4094/users/655a2d89f6e440bc8634c5d8aab17ccf/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:38:19.383780 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:38:19.431737 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e317a92e-d386-40a2-9252-3ab17a883a5c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:19.433424 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e317a92e-d386-40a2-9252-3ab17a883a5c service swift] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:19.444332 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 959/1910] 10.222.0.70 () {60 vars in 1384 bytes} [Thu Oct 27 09:38:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 27 09:38:20.403539 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcafb55-6a45-41b8-a760-a1fd1e8694a5 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:20.404112 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcafb55-6a45-41b8-a760-a1fd1e8694a5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:20.404511 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcafb55-6a45-41b8-a760-a1fd1e8694a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:20.441568 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-cfcafb55-6a45-41b8-a760-a1fd1e8694a5 None None] MFA Rules not processed for user `7f389510a6a044d8a7487561b169544b`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:38:20.442820 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-cfcafb55-6a45-41b8-a760-a1fd1e8694a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:20.554474 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 952/1911] 10.222.0.70 () {58 vars in 967 bytes} [Thu Oct 27 09:38:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:38:20.572923 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e317a92e-d386-40a2-9252-3ab17a883a5c service swift] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:20.584339 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e317a92e-d386-40a2-9252-3ab17a883a5c service swift] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:20.700852 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e80b80-26b8-4bbe-b1e9-00a8c76b8931 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eecf79ea4700496c89b1a67c614b34d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:20.701579 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e80b80-26b8-4bbe-b1e9-00a8c76b8931 service swift] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:20.701954 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e80b80-26b8-4bbe-b1e9-00a8c76b8931 service swift] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:20.702317 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e80b80-26b8-4bbe-b1e9-00a8c76b8931 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:20.713119 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e80b80-26b8-4bbe-b1e9-00a8c76b8931 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:20.715985 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e80b80-26b8-4bbe-b1e9-00a8c76b8931 service swift] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:20.726094 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 960/1912] 10.222.0.70 () {60 vars in 1384 bytes} [Thu Oct 27 09:38:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 27 09:38:21.159596 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e9737-1855-4c9a-881d-9d4e2d9ed1cb None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:21.160006 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e9737-1855-4c9a-881d-9d4e2d9ed1cb None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:21.160417 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e9737-1855-4c9a-881d-9d4e2d9ed1cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:21.200578 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-6a1e9737-1855-4c9a-881d-9d4e2d9ed1cb None None] MFA Rules not processed for user `7f389510a6a044d8a7487561b169544b`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:38:21.203378 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6a1e9737-1855-4c9a-881d-9d4e2d9ed1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:21.223482 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 953/1913] 10.222.0.70 () {58 vars in 1009 bytes} [Thu Oct 27 09:38:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:38:21.235241 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e80b80-26b8-4bbe-b1e9-00a8c76b8931 service swift] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:21.259891 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e80b80-26b8-4bbe-b1e9-00a8c76b8931 service swift] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:21.284373 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde0490f-8919-412a-aa52-49b387e24bfd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eecf79ea4700496c89b1a67c614b34d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:21.285421 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0490f-8919-412a-aa52-49b387e24bfd service swift] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:21.285851 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0490f-8919-412a-aa52-49b387e24bfd service swift] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:21.301741 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0490f-8919-412a-aa52-49b387e24bfd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:21.337645 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde0490f-8919-412a-aa52-49b387e24bfd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:21.339819 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde0490f-8919-412a-aa52-49b387e24bfd service swift] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:21.359277 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 961/1914] 10.222.0.70 () {60 vars in 1384 bytes} [Thu Oct 27 09:38:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 27 09:38:22.039034 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ed998-69ce-4ae5-a25a-f4fdcdf0588a None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:22.039571 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ed998-69ce-4ae5-a25a-f4fdcdf0588a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:22.039571 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ed998-69ce-4ae5-a25a-f4fdcdf0588a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:22.085756 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a61ed998-69ce-4ae5-a25a-f4fdcdf0588a None None] MFA Rules not processed for user `7f389510a6a044d8a7487561b169544b`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:38:22.086983 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a61ed998-69ce-4ae5-a25a-f4fdcdf0588a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:22.120192 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 954/1915] 10.222.0.70 () {58 vars in 967 bytes} [Thu Oct 27 09:38:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:38:22.137145 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde0490f-8919-412a-aa52-49b387e24bfd service swift] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:22.150775 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde0490f-8919-412a-aa52-49b387e24bfd service swift] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:22.170079 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad5fab6-5bb5-4be8-82ab-fcd41677b28c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eecf79ea4700496c89b1a67c614b34d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:22.171060 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad5fab6-5bb5-4be8-82ab-fcd41677b28c service swift] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:22.171501 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad5fab6-5bb5-4be8-82ab-fcd41677b28c service swift] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:22.171916 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad5fab6-5bb5-4be8-82ab-fcd41677b28c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:22.201738 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad5fab6-5bb5-4be8-82ab-fcd41677b28c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:22.203642 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad5fab6-5bb5-4be8-82ab-fcd41677b28c service swift] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:22.227885 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 962/1916] 10.222.0.70 () {60 vars in 1384 bytes} [Thu Oct 27 09:38:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 27 09:38:25.035309 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ed998-69ce-4ae5-a25a-f4fdcdf0588a None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:25.044653 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a61ed998-69ce-4ae5-a25a-f4fdcdf0588a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:25.054760 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:38:25.133191 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ed998-69ce-4ae5-a25a-f4fdcdf0588a None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:25.138384 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695892a9-15d1-4ec3-9962-5457fffc9e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:25.139321 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695892a9-15d1-4ec3-9962-5457fffc9e38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:25.139700 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695892a9-15d1-4ec3-9962-5457fffc9e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:25.141409 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695892a9-15d1-4ec3-9962-5457fffc9e38 admin admin] PATH_INFO: `/v3/users/c625f65762224ff9a799243ef0b527fc` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:25.142826 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695892a9-15d1-4ec3-9962-5457fffc9e38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c625f65762224ff9a799243ef0b527fc)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:25.145100 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695892a9-15d1-4ec3-9962-5457fffc9e38 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:25.213300 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-695892a9-15d1-4ec3-9962-5457fffc9e38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c625f65762224ff9a799243ef0b527fc'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:25.226063 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-695892a9-15d1-4ec3-9962-5457fffc9e38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c625f65762224ff9a799243ef0b527fc'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:25.241267 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-695892a9-15d1-4ec3-9962-5457fffc9e38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c625f65762224ff9a799243ef0b527fc'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:25.251009 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-695892a9-15d1-4ec3-9962-5457fffc9e38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c625f65762224ff9a799243ef0b527fc'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:25.251908 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-695892a9-15d1-4ec3-9962-5457fffc9e38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c625f65762224ff9a799243ef0b527fc'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:25.262416 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 955/1917] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:38:25 2022] DELETE /identity/v3/users/c625f65762224ff9a799243ef0b527fc => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:38:25.264278 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdaed81-408c-41a2-8dfc-12d886424ccd None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:25.264798 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdaed81-408c-41a2-8dfc-12d886424ccd None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:25.264881 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdaed81-408c-41a2-8dfc-12d886424ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:25.287089 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-bfdaed81-408c-41a2-8dfc-12d886424ccd None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:38:25.288563 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-bfdaed81-408c-41a2-8dfc-12d886424ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:25.377019 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 963/1918] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:38:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:38:25.385932 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695892a9-15d1-4ec3-9962-5457fffc9e38 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:25.388022 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-695892a9-15d1-4ec3-9962-5457fffc9e38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:25.462449 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695892a9-15d1-4ec3-9962-5457fffc9e38 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:25.468369 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e2ab77-ad46-4c14-a269-48ae72af524f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:25.469078 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2ab77-ad46-4c14-a269-48ae72af524f service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:25.469364 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2ab77-ad46-4c14-a269-48ae72af524f service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:25.469636 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2ab77-ad46-4c14-a269-48ae72af524f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:25.480938 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e2ab77-ad46-4c14-a269-48ae72af524f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:25.482402 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e2ab77-ad46-4c14-a269-48ae72af524f service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:25.510222 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 956/1919] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:38:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:38:25.721904 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdaed81-408c-41a2-8dfc-12d886424ccd None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:25.724555 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-bfdaed81-408c-41a2-8dfc-12d886424ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:25.756247 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdaed81-408c-41a2-8dfc-12d886424ccd None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:25.772390 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd43f04a-ba17-4ee9-a134-5ce84b1100a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:25.773288 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43f04a-ba17-4ee9-a134-5ce84b1100a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:25.773694 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43f04a-ba17-4ee9-a134-5ce84b1100a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:25.774037 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43f04a-ba17-4ee9-a134-5ce84b1100a1 admin admin] PATH_INFO: `/v3/projects/e694ff25532a41bfac762f1e11e3869c` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:25.775838 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd43f04a-ba17-4ee9-a134-5ce84b1100a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e694ff25532a41bfac762f1e11e3869c)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:25.779677 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd43f04a-ba17-4ee9-a134-5ce84b1100a1 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:25.823547 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:38:25.892353 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-fd43f04a-ba17-4ee9-a134-5ce84b1100a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e694ff25532a41bfac762f1e11e3869c', 'request_id': 'req-fd43f04a-ba17-4ee9-a134-5ce84b1100a1'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:25.919800 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-fd43f04a-ba17-4ee9-a134-5ce84b1100a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e694ff25532a41bfac762f1e11e3869c', 'request_id': 'req-fd43f04a-ba17-4ee9-a134-5ce84b1100a1'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:25.934769 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-fd43f04a-ba17-4ee9-a134-5ce84b1100a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e694ff25532a41bfac762f1e11e3869c', 'request_id': 'req-fd43f04a-ba17-4ee9-a134-5ce84b1100a1'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:25.969145 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-fd43f04a-ba17-4ee9-a134-5ce84b1100a1 admin admin] The token cache is being invalidate because project e694ff25532a41bfac762f1e11e3869c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:38:25.969820 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-fd43f04a-ba17-4ee9-a134-5ce84b1100a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:25.970652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-fd43f04a-ba17-4ee9-a134-5ce84b1100a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:25.982607 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 964/1920] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:38:25 2022] DELETE /identity/v3/projects/e694ff25532a41bfac762f1e11e3869c => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:38:26.796376 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d116e9-d8af-46ca-9eeb-12f98617cae9 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:26.796885 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d116e9-d8af-46ca-9eeb-12f98617cae9 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:26.796885 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d116e9-d8af-46ca-9eeb-12f98617cae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:26.828351 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-e7d116e9-d8af-46ca-9eeb-12f98617cae9 None None] MFA Rules not processed for user `655a2d89f6e440bc8634c5d8aab17ccf`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:38:26.829880 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e7d116e9-d8af-46ca-9eeb-12f98617cae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:26.844352 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:38:26.905062 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 957/1921] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:38:26 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:38:26.918538 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd43f04a-ba17-4ee9-a134-5ce84b1100a1 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:26.919971 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-fd43f04a-ba17-4ee9-a134-5ce84b1100a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:26.993455 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd43f04a-ba17-4ee9-a134-5ce84b1100a1 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:26.999493 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35257dd-9c12-4555-9927-52d399de6d52 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe4a6e4b6184826b30ac4fc13ad9b1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:27.000044 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35257dd-9c12-4555-9927-52d399de6d52 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:27.000321 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35257dd-9c12-4555-9927-52d399de6d52 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:27.000628 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35257dd-9c12-4555-9927-52d399de6d52 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:27.002364 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f35257dd-9c12-4555-9927-52d399de6d52 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:27.061641 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35257dd-9c12-4555-9927-52d399de6d52 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:27.063223 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35257dd-9c12-4555-9927-52d399de6d52 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:27.072018 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 965/1922] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:38:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:38:28.181486 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99888941-6f84-4efd-9707-d49adfb6b717 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:28.181959 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99888941-6f84-4efd-9707-d49adfb6b717 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:28.182052 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99888941-6f84-4efd-9707-d49adfb6b717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:28.201925 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-99888941-6f84-4efd-9707-d49adfb6b717 None None] MFA Rules not processed for user `0968b48cb99d4168b36a2602bd87f788`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:38:28.203124 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-99888941-6f84-4efd-9707-d49adfb6b717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:28.210334 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 958/1923] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:38:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:38:37.068173 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=71,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:38:38.196563 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=25 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:38:41.239210 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35257dd-9c12-4555-9927-52d399de6d52 service nova] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:41.240643 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f35257dd-9c12-4555-9927-52d399de6d52 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:41.312201 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35257dd-9c12-4555-9927-52d399de6d52 service nova] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:41.316792 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126e1cd9-c9ca-4583-ae5f-dc6210526c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:41.317434 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126e1cd9-c9ca-4583-ae5f-dc6210526c9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:41.317711 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126e1cd9-c9ca-4583-ae5f-dc6210526c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:41.317984 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126e1cd9-c9ca-4583-ae5f-dc6210526c9d admin admin] PATH_INFO: `/v3/users/0c33508f209b40628451de85db4e71e5` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:41.319092 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126e1cd9-c9ca-4583-ae5f-dc6210526c9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c33508f209b40628451de85db4e71e5)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:41.321079 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126e1cd9-c9ca-4583-ae5f-dc6210526c9d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:41.372228 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-126e1cd9-c9ca-4583-ae5f-dc6210526c9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c33508f209b40628451de85db4e71e5'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:41.379129 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-126e1cd9-c9ca-4583-ae5f-dc6210526c9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c33508f209b40628451de85db4e71e5'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:41.394636 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-126e1cd9-c9ca-4583-ae5f-dc6210526c9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c33508f209b40628451de85db4e71e5'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:41.402472 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-126e1cd9-c9ca-4583-ae5f-dc6210526c9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c33508f209b40628451de85db4e71e5'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:41.403153 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-126e1cd9-c9ca-4583-ae5f-dc6210526c9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c33508f209b40628451de85db4e71e5'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:41.412973 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 966/1924] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:38:41 2022] DELETE /identity/v3/users/0c33508f209b40628451de85db4e71e5 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:38:41.415396 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4265d1ac-6b94-41e2-84bc-6d59770d5164 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:41.416946 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4265d1ac-6b94-41e2-84bc-6d59770d5164 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:41.417237 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4265d1ac-6b94-41e2-84bc-6d59770d5164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:41.439134 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-4265d1ac-6b94-41e2-84bc-6d59770d5164 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:38:41.440355 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4265d1ac-6b94-41e2-84bc-6d59770d5164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:41.513984 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 959/1925] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:38:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:38:41.523567 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126e1cd9-c9ca-4583-ae5f-dc6210526c9d admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:41.525002 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-126e1cd9-c9ca-4583-ae5f-dc6210526c9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:41.583135 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126e1cd9-c9ca-4583-ae5f-dc6210526c9d admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:41.586749 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2039a992-15e6-469f-a10a-cf57aa2f3f4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:41.587356 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2039a992-15e6-469f-a10a-cf57aa2f3f4b service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:41.587629 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2039a992-15e6-469f-a10a-cf57aa2f3f4b service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:41.587896 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2039a992-15e6-469f-a10a-cf57aa2f3f4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:41.596658 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2039a992-15e6-469f-a10a-cf57aa2f3f4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:41.598130 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2039a992-15e6-469f-a10a-cf57aa2f3f4b service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:41.607256 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 967/1926] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:38:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:38:41.891905 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4265d1ac-6b94-41e2-84bc-6d59770d5164 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:41.893599 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4265d1ac-6b94-41e2-84bc-6d59770d5164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:41.913175 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4265d1ac-6b94-41e2-84bc-6d59770d5164 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:41.920543 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3defff-357f-40f8-8a31-95bd2ffb5a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:41.921258 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3defff-357f-40f8-8a31-95bd2ffb5a71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:41.921605 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3defff-357f-40f8-8a31-95bd2ffb5a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:41.922069 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3defff-357f-40f8-8a31-95bd2ffb5a71 admin admin] PATH_INFO: `/v3/projects/2c69f633a97d49caa0ae0123eacacd4c` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:41.923627 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3defff-357f-40f8-8a31-95bd2ffb5a71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c69f633a97d49caa0ae0123eacacd4c)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:41.925697 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3defff-357f-40f8-8a31-95bd2ffb5a71 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:41.983160 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-2d3defff-357f-40f8-8a31-95bd2ffb5a71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c69f633a97d49caa0ae0123eacacd4c', 'request_id': 'req-2d3defff-357f-40f8-8a31-95bd2ffb5a71'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:41.989246 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-2d3defff-357f-40f8-8a31-95bd2ffb5a71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c69f633a97d49caa0ae0123eacacd4c', 'request_id': 'req-2d3defff-357f-40f8-8a31-95bd2ffb5a71'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:41.994666 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-2d3defff-357f-40f8-8a31-95bd2ffb5a71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c69f633a97d49caa0ae0123eacacd4c', 'request_id': 'req-2d3defff-357f-40f8-8a31-95bd2ffb5a71'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:42.007114 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-2d3defff-357f-40f8-8a31-95bd2ffb5a71 admin admin] The token cache is being invalidate because project 2c69f633a97d49caa0ae0123eacacd4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:38:42.007394 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-2d3defff-357f-40f8-8a31-95bd2ffb5a71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:42.008041 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-2d3defff-357f-40f8-8a31-95bd2ffb5a71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:38:42.009719 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 960/1927] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:38:41 2022] DELETE /identity/v3/projects/2c69f633a97d49caa0ae0123eacacd4c => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:38:42.015570 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357c1e4-5ed1-4d32-a5eb-e13577eb731e None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:42.015933 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357c1e4-5ed1-4d32-a5eb-e13577eb731e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:42.016270 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357c1e4-5ed1-4d32-a5eb-e13577eb731e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:42.038673 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-c357c1e4-5ed1-4d32-a5eb-e13577eb731e None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:38:42.040113 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c357c1e4-5ed1-4d32-a5eb-e13577eb731e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:42.058063 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:38:42.103869 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 968/1928] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:38:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:38:42.112433 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f614e16-14fc-4d8c-869d-4e0e5c88f0a4 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:42.112774 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f614e16-14fc-4d8c-869d-4e0e5c88f0a4 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:42.113050 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f614e16-14fc-4d8c-869d-4e0e5c88f0a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:42.133077 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-4f614e16-14fc-4d8c-869d-4e0e5c88f0a4 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:38:42.134290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4f614e16-14fc-4d8c-869d-4e0e5c88f0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:42.148675 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 961/1929] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:38:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:38:42.150490 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c357c1e4-5ed1-4d32-a5eb-e13577eb731e None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:42.160109 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c357c1e4-5ed1-4d32-a5eb-e13577eb731e None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:42.164926 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7da34e8-f27f-4098-b8be-3a520e66060e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:42.165641 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7da34e8-f27f-4098-b8be-3a520e66060e admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:42.165907 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7da34e8-f27f-4098-b8be-3a520e66060e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:42.166168 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7da34e8-f27f-4098-b8be-3a520e66060e admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:42.166814 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7da34e8-f27f-4098-b8be-3a520e66060e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:38:42.167132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7da34e8-f27f-4098-b8be-3a520e66060e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:42.168826 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7da34e8-f27f-4098-b8be-3a520e66060e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:42.181190 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 969/1930] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:38:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:38:42.185936 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f614e16-14fc-4d8c-869d-4e0e5c88f0a4 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:42.195020 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f614e16-14fc-4d8c-869d-4e0e5c88f0a4 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:42.200197 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe380c5-7d60-44ae-bdce-f8840e8d2e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:42.200846 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe380c5-7d60-44ae-bdce-f8840e8d2e56 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:42.201120 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe380c5-7d60-44ae-bdce-f8840e8d2e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:42.201386 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe380c5-7d60-44ae-bdce-f8840e8d2e56 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:42.201996 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe380c5-7d60-44ae-bdce-f8840e8d2e56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:42.204257 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe380c5-7d60-44ae-bdce-f8840e8d2e56 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:42.232905 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 962/1931] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:38:42 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:38:42.237195 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7da34e8-f27f-4098-b8be-3a520e66060e admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:42.248309 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7da34e8-f27f-4098-b8be-3a520e66060e admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:42.305579 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d46db62-e467-4420-97b3-3e2a3eea42c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:42.306321 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d46db62-e467-4420-97b3-3e2a3eea42c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:42.306602 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d46db62-e467-4420-97b3-3e2a3eea42c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:42.306880 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d46db62-e467-4420-97b3-3e2a3eea42c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:42.307505 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d46db62-e467-4420-97b3-3e2a3eea42c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:42.311151 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d46db62-e467-4420-97b3-3e2a3eea42c3 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:42.332887 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 970/1932] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:38:42 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:38:42.337117 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe380c5-7d60-44ae-bdce-f8840e8d2e56 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:42.348302 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe380c5-7d60-44ae-bdce-f8840e8d2e56 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:42.353185 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c1fb05-adf9-4f02-bd6c-a52d0c231413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:42.353883 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c1fb05-adf9-4f02-bd6c-a52d0c231413 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:42.354233 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c1fb05-adf9-4f02-bd6c-a52d0c231413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:42.354607 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c1fb05-adf9-4f02-bd6c-a52d0c231413 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:42.355447 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c1fb05-adf9-4f02-bd6c-a52d0c231413 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:42.357649 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c1fb05-adf9-4f02-bd6c-a52d0c231413 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:42.374541 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 963/1933] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:38:42 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 09:38:42.378612 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d46db62-e467-4420-97b3-3e2a3eea42c3 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:42.388480 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d46db62-e467-4420-97b3-3e2a3eea42c3 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:42.393282 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec023d4-f57f-49d1-ad50-1efd72117ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:42.393897 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec023d4-f57f-49d1-ad50-1efd72117ec3 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:42.394172 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec023d4-f57f-49d1-ad50-1efd72117ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:42.394439 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec023d4-f57f-49d1-ad50-1efd72117ec3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:42.395062 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec023d4-f57f-49d1-ad50-1efd72117ec3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:42.396664 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec023d4-f57f-49d1-ad50-1efd72117ec3 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:42.408024 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 971/1934] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:38:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:38:42.412227 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c1fb05-adf9-4f02-bd6c-a52d0c231413 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:42.423847 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c1fb05-adf9-4f02-bd6c-a52d0c231413 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:42.428896 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ed087d-4b6a-497f-95e7-7a06bc42d02f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:42.429590 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed087d-4b6a-497f-95e7-7a06bc42d02f admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:42.429868 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed087d-4b6a-497f-95e7-7a06bc42d02f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:42.430140 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed087d-4b6a-497f-95e7-7a06bc42d02f admin admin] PATH_INFO: `/v3/projects/f169a47a4c9a4c60b87cedcd77c102c8/users/51aba36d56c94e2fa351aa098dc0a962/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:42.445084 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ed087d-4b6a-497f-95e7-7a06bc42d02f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f169a47a4c9a4c60b87cedcd77c102c8, user_id=51aba36d56c94e2fa351aa098dc0a962, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:42.447720 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ed087d-4b6a-497f-95e7-7a06bc42d02f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:42.461506 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 964/1935] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:38:42 2022] PUT /identity/v3/projects/f169a47a4c9a4c60b87cedcd77c102c8/users/51aba36d56c94e2fa351aa098dc0a962/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:38:42.467156 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c22485-f317-4e70-91e4-afdb0a660bf9 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:42.467619 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c22485-f317-4e70-91e4-afdb0a660bf9 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:42.467994 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c22485-f317-4e70-91e4-afdb0a660bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:42.490237 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f6c22485-f317-4e70-91e4-afdb0a660bf9 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:38:42.491849 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f6c22485-f317-4e70-91e4-afdb0a660bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:42.583110 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 972/1936] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:38:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:38:42.593607 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ed087d-4b6a-497f-95e7-7a06bc42d02f admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:42.595611 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-02ed087d-4b6a-497f-95e7-7a06bc42d02f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:42.619158 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:38:42.659587 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ed087d-4b6a-497f-95e7-7a06bc42d02f admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:42.663372 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ace047-4dfb-48df-ae9b-ded30bd124a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:42.663977 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ace047-4dfb-48df-ae9b-ded30bd124a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:42.664371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ace047-4dfb-48df-ae9b-ded30bd124a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:42.664699 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ace047-4dfb-48df-ae9b-ded30bd124a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:42.674758 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ace047-4dfb-48df-ae9b-ded30bd124a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:42.676919 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ace047-4dfb-48df-ae9b-ded30bd124a4 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:42.687958 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 965/1937] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:38:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:38:47.720143 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0cdd1-9308-4ced-9357-653fc10a7788 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:47.720711 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0cdd1-9308-4ced-9357-653fc10a7788 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:47.720904 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0cdd1-9308-4ced-9357-653fc10a7788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:47.754961 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-0cc0cdd1-9308-4ced-9357-653fc10a7788 None None] MFA Rules not processed for user `51aba36d56c94e2fa351aa098dc0a962`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:38:47.756230 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0cc0cdd1-9308-4ced-9357-653fc10a7788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:47.783803 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:38:47.845928 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 973/1938] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:38:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:38:47.850802 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ace047-4dfb-48df-ae9b-ded30bd124a4 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:47.863373 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ace047-4dfb-48df-ae9b-ded30bd124a4 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:47.868355 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb563fa5-b9b9-4db6-be06-85f4a776f4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:47.869760 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb563fa5-b9b9-4db6-be06-85f4a776f4e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:47.870209 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb563fa5-b9b9-4db6-be06-85f4a776f4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:47.870731 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb563fa5-b9b9-4db6-be06-85f4a776f4e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:47.871572 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb563fa5-b9b9-4db6-be06-85f4a776f4e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:47.881258 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb563fa5-b9b9-4db6-be06-85f4a776f4e0 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:47.924573 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 966/1939] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:38:47 2022] POST /identity/v3/projects => generated 383 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:38:47.926588 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc0cdd1-9308-4ced-9357-653fc10a7788 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:47.941900 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc0cdd1-9308-4ced-9357-653fc10a7788 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:48.049877 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ae5330-b77a-42ab-a9e9-cd573aeed172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:48.050971 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae5330-b77a-42ab-a9e9-cd573aeed172 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:48.052115 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae5330-b77a-42ab-a9e9-cd573aeed172 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:48.052264 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae5330-b77a-42ab-a9e9-cd573aeed172 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:48.055709 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ae5330-b77a-42ab-a9e9-cd573aeed172 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:48.060092 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ae5330-b77a-42ab-a9e9-cd573aeed172 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:48.095944 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 974/1940] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:38:47 2022] POST /identity/v3/users => generated 298 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:38:48.097312 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb563fa5-b9b9-4db6-be06-85f4a776f4e0 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:48.129381 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb563fa5-b9b9-4db6-be06-85f4a776f4e0 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:48.135810 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff9dae2-11af-4230-a819-09f078fb31e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:48.136469 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9dae2-11af-4230-a819-09f078fb31e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:48.136817 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9dae2-11af-4230-a819-09f078fb31e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:48.137273 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9dae2-11af-4230-a819-09f078fb31e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:48.137938 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff9dae2-11af-4230-a819-09f078fb31e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:48.139565 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff9dae2-11af-4230-a819-09f078fb31e8 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:48.151376 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 967/1941] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:38:48 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:38:48.156281 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ae5330-b77a-42ab-a9e9-cd573aeed172 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:48.172343 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ae5330-b77a-42ab-a9e9-cd573aeed172 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:48.181115 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f69c22f-2ebb-4424-aa72-9e79b251911e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:48.181910 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69c22f-2ebb-4424-aa72-9e79b251911e admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:48.182117 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69c22f-2ebb-4424-aa72-9e79b251911e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:48.182363 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69c22f-2ebb-4424-aa72-9e79b251911e admin admin] PATH_INFO: `/v3/projects/39b5b195f9b04586b6e8e4a33271619c/users/81100e3911bc4ac3a819e562d3dc97e0/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:48.201016 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f69c22f-2ebb-4424-aa72-9e79b251911e admin admin] RBAC: Authorizing `identity:create_grant(project_id=39b5b195f9b04586b6e8e4a33271619c, user_id=81100e3911bc4ac3a819e562d3dc97e0, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:48.203569 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f69c22f-2ebb-4424-aa72-9e79b251911e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:48.223351 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 975/1942] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:38:48 2022] PUT /identity/v3/projects/39b5b195f9b04586b6e8e4a33271619c/users/81100e3911bc4ac3a819e562d3dc97e0/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:38:48.227356 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff9dae2-11af-4230-a819-09f078fb31e8 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:48.240950 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff9dae2-11af-4230-a819-09f078fb31e8 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:48.313367 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2852ace2-dc02-4d22-8517-6850fcf32dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:48.314271 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852ace2-dc02-4d22-8517-6850fcf32dfa admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:48.314271 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852ace2-dc02-4d22-8517-6850fcf32dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:48.314554 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852ace2-dc02-4d22-8517-6850fcf32dfa admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:48.315237 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2852ace2-dc02-4d22-8517-6850fcf32dfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:48.316970 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2852ace2-dc02-4d22-8517-6850fcf32dfa admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:48.328417 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 968/1943] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:38:48 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:38:48.333109 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f69c22f-2ebb-4424-aa72-9e79b251911e admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:48.344106 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f69c22f-2ebb-4424-aa72-9e79b251911e admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:48.349004 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aacb8f-44e1-4e65-9911-bf726842e9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:48.349674 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aacb8f-44e1-4e65-9911-bf726842e9a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:48.349977 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aacb8f-44e1-4e65-9911-bf726842e9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:48.350375 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aacb8f-44e1-4e65-9911-bf726842e9a3 admin admin] PATH_INFO: `/v3/projects/39b5b195f9b04586b6e8e4a33271619c/users/81100e3911bc4ac3a819e562d3dc97e0/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:48.352165 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aacb8f-44e1-4e65-9911-bf726842e9a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39b5b195f9b04586b6e8e4a33271619c, user_id=81100e3911bc4ac3a819e562d3dc97e0, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:48.354844 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aacb8f-44e1-4e65-9911-bf726842e9a3 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:48.365953 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 976/1944] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:38:48 2022] PUT /identity/v3/projects/39b5b195f9b04586b6e8e4a33271619c/users/81100e3911bc4ac3a819e562d3dc97e0/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:38:53.889700 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2301a740-b3c1-42d2-b826-dd5f2252b645 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:53.890183 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2301a740-b3c1-42d2-b826-dd5f2252b645 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:53.890517 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2301a740-b3c1-42d2-b826-dd5f2252b645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:53.926368 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-2301a740-b3c1-42d2-b826-dd5f2252b645 None None] MFA Rules not processed for user `81100e3911bc4ac3a819e562d3dc97e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:38:53.927618 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2301a740-b3c1-42d2-b826-dd5f2252b645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:54.041064 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 969/1945] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:38:53 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:38:54.052861 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aacb8f-44e1-4e65-9911-bf726842e9a3 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:54.054383 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-95aacb8f-44e1-4e65-9911-bf726842e9a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:54.151661 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aacb8f-44e1-4e65-9911-bf726842e9a3 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:54.157169 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8421820c-8630-46e6-8b72-62f73dc33d1b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72b0538f0e42446887b1d95b2fb41fb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:54.157985 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421820c-8630-46e6-8b72-62f73dc33d1b service glance] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:54.158259 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421820c-8630-46e6-8b72-62f73dc33d1b service glance] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:54.158530 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421820c-8630-46e6-8b72-62f73dc33d1b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:54.193221 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:38:54.265576 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8421820c-8630-46e6-8b72-62f73dc33d1b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:54.267367 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8421820c-8630-46e6-8b72-62f73dc33d1b service glance] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:54.277802 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 977/1946] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:38:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:38:54.524159 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac314ed-a41e-479c-8664-c308dfd6a67b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:54.524356 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac314ed-a41e-479c-8664-c308dfd6a67b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:54.524657 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac314ed-a41e-479c-8664-c308dfd6a67b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:54.570351 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-5ac314ed-a41e-479c-8664-c308dfd6a67b None None] MFA Rules not processed for user `7f389510a6a044d8a7487561b169544b`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:38:54.571594 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5ac314ed-a41e-479c-8664-c308dfd6a67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:54.581999 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:38:54.680685 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 970/1947] 10.222.0.70 () {58 vars in 1009 bytes} [Thu Oct 27 09:38:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:38:54.695827 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8421820c-8630-46e6-8b72-62f73dc33d1b service glance] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:38:54.697959 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8421820c-8630-46e6-8b72-62f73dc33d1b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:38:54.805655 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8421820c-8630-46e6-8b72-62f73dc33d1b service glance] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:38:54.814263 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde4189-2141-44cb-b1e3-2a9a851f763d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eecf79ea4700496c89b1a67c614b34d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:38:54.815113 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde4189-2141-44cb-b1e3-2a9a851f763d service swift] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:38:54.815553 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde4189-2141-44cb-b1e3-2a9a851f763d service swift] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:38:54.815824 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde4189-2141-44cb-b1e3-2a9a851f763d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:38:54.825487 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dde4189-2141-44cb-b1e3-2a9a851f763d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:38:54.827131 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dde4189-2141-44cb-b1e3-2a9a851f763d service swift] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:38:54.837759 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 978/1948] 10.222.0.70 () {60 vars in 1384 bytes} [Thu Oct 27 09:38:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 27 09:39:04.065860 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f5fad-a719-491b-943b-09ca100abd0b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:39:04.066921 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f5fad-a719-491b-943b-09ca100abd0b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:39:04.067410 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f5fad-a719-491b-943b-09ca100abd0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:39:04.092804 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b72f5fad-a719-491b-943b-09ca100abd0b None None] MFA Rules not processed for user `7f389510a6a044d8a7487561b169544b`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:39:04.094662 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b72f5fad-a719-491b-943b-09ca100abd0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:39:04.103008 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 971/1949] 10.222.0.70 () {58 vars in 1009 bytes} [Thu Oct 27 09:39:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:39:04.118358 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde4189-2141-44cb-b1e3-2a9a851f763d service swift] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:39:04.143204 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde4189-2141-44cb-b1e3-2a9a851f763d service swift] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:39:04.152293 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1f99f2-36ab-407c-9609-570251dd8b07 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eecf79ea4700496c89b1a67c614b34d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:39:04.153948 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f99f2-36ab-407c-9609-570251dd8b07 service swift] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:39:04.154875 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f99f2-36ab-407c-9609-570251dd8b07 service swift] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:39:04.156454 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f99f2-36ab-407c-9609-570251dd8b07 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:39:04.168461 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1f99f2-36ab-407c-9609-570251dd8b07 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:39:04.170945 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1f99f2-36ab-407c-9609-570251dd8b07 service swift] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:39:04.191135 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 979/1950] 10.222.0.70 () {60 vars in 1384 bytes} [Thu Oct 27 09:39:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 27 09:39:04.476489 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c28e38-c79b-4ba2-b8c6-7b30d1ec79c0 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:39:04.478078 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c28e38-c79b-4ba2-b8c6-7b30d1ec79c0 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:39:04.478704 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c28e38-c79b-4ba2-b8c6-7b30d1ec79c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:39:04.527034 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-89c28e38-c79b-4ba2-b8c6-7b30d1ec79c0 None None] MFA Rules not processed for user `20c65ed67cd8403489949c4dea572d01`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:39:04.529838 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-89c28e38-c79b-4ba2-b8c6-7b30d1ec79c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:39:04.658357 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 972/1951] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:39:04 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:39:07.621726 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e2a8f3-2e96-4130-ba90-3e292749fc7f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:39:07.622276 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e2a8f3-2e96-4130-ba90-3e292749fc7f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:39:07.622276 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e2a8f3-2e96-4130-ba90-3e292749fc7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:39:07.665746 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-56e2a8f3-2e96-4130-ba90-3e292749fc7f None None] MFA Rules not processed for user `51aba36d56c94e2fa351aa098dc0a962`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:39:07.668505 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-56e2a8f3-2e96-4130-ba90-3e292749fc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:39:07.684853 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 980/1952] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:39:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:39:14.635970 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=48 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:39:17.647170 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=57 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:39:19.135502 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c88f51-bf7d-429e-8098-cc4c1bbb6bb3 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:39:19.136704 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c88f51-bf7d-429e-8098-cc4c1bbb6bb3 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:39:19.137411 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c88f51-bf7d-429e-8098-cc4c1bbb6bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:39:19.165544 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-02c88f51-bf7d-429e-8098-cc4c1bbb6bb3 None None] MFA Rules not processed for user `7f389510a6a044d8a7487561b169544b`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:39:19.167892 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-02c88f51-bf7d-429e-8098-cc4c1bbb6bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:39:19.178134 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 973/1953] 10.222.0.70 () {58 vars in 1009 bytes} [Thu Oct 27 09:39:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:39:19.215807 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e2a8f3-2e96-4130-ba90-3e292749fc7f None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:39:19.230842 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e2a8f3-2e96-4130-ba90-3e292749fc7f None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:39:19.236762 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf0fafe-b8a9-481b-81c2-452270e922af service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eecf79ea4700496c89b1a67c614b34d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:39:19.237701 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf0fafe-b8a9-481b-81c2-452270e922af service swift] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:39:19.238151 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf0fafe-b8a9-481b-81c2-452270e922af service swift] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:39:19.238599 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf0fafe-b8a9-481b-81c2-452270e922af service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:39:19.251891 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf0fafe-b8a9-481b-81c2-452270e922af service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:39:19.254903 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf0fafe-b8a9-481b-81c2-452270e922af service swift] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:39:19.293220 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 981/1954] 10.222.0.70 () {60 vars in 1384 bytes} [Thu Oct 27 09:39:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 27 09:39:29.160058 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:39:29.263992 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=6 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:39:32.505140 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c88f51-bf7d-429e-8098-cc4c1bbb6bb3 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:39:32.506582 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-02c88f51-bf7d-429e-8098-cc4c1bbb6bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:39:32.561719 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c88f51-bf7d-429e-8098-cc4c1bbb6bb3 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:39:32.565407 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e60b3e-c32b-4242-a02c-8115a8640c32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:39:32.566011 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e60b3e-c32b-4242-a02c-8115a8640c32 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:39:32.566285 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e60b3e-c32b-4242-a02c-8115a8640c32 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:39:32.566552 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e60b3e-c32b-4242-a02c-8115a8640c32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:39:32.568240 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-52e60b3e-c32b-4242-a02c-8115a8640c32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:39:32.629988 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e60b3e-c32b-4242-a02c-8115a8640c32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:39:32.631651 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e60b3e-c32b-4242-a02c-8115a8640c32 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:39:32.641512 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 974/1955] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:39:32 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:39:36.745198 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b6948b-9c41-4fba-96cc-4f5f65d72dc7 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:39:36.746267 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b6948b-9c41-4fba-96cc-4f5f65d72dc7 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:39:36.746573 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b6948b-9c41-4fba-96cc-4f5f65d72dc7 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:39:36.748308 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 982/1956] 10.222.0.70 () {54 vars in 921 bytes} [Thu Oct 27 09:39:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:39:36.753808 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6938dd-74dc-4ca8-a16b-21d593e022fd None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:39:36.754251 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6938dd-74dc-4ca8-a16b-21d593e022fd None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:39:36.754497 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6938dd-74dc-4ca8-a16b-21d593e022fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:39:36.774897 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-eb6938dd-74dc-4ca8-a16b-21d593e022fd None None] MFA Rules not processed for user `dbe4a6e4b6184826b30ac4fc13ad9b1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:39:36.776113 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-eb6938dd-74dc-4ca8-a16b-21d593e022fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:39:36.853610 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 975/1957] 10.222.0.70 () {58 vars in 1018 bytes} [Thu Oct 27 09:39:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:39:36.868903 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b6948b-9c41-4fba-96cc-4f5f65d72dc7 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:39:36.884811 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b6948b-9c41-4fba-96cc-4f5f65d72dc7 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:39:36.884943 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c244812b-5258-4f4d-854d-13468df00b4c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72b0538f0e42446887b1d95b2fb41fb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:39:36.885534 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244812b-5258-4f4d-854d-13468df00b4c service glance] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:39:36.885809 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244812b-5258-4f4d-854d-13468df00b4c service glance] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:39:36.886076 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244812b-5258-4f4d-854d-13468df00b4c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:39:36.897236 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c244812b-5258-4f4d-854d-13468df00b4c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:39:36.898652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c244812b-5258-4f4d-854d-13468df00b4c service glance] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:39:36.909451 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 983/1958] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:39:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:39:36.981475 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc1ea7-34f0-4ddf-9e2a-3c793b345892 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:39:36.981759 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc1ea7-34f0-4ddf-9e2a-3c793b345892 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:39:36.982030 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc1ea7-34f0-4ddf-9e2a-3c793b345892 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:39:36.983367 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 976/1959] 10.222.0.70 () {54 vars in 921 bytes} [Thu Oct 27 09:39:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:39:36.988159 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539233b-d0fe-48b3-bbbb-d6321ddf296c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:39:36.988462 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539233b-d0fe-48b3-bbbb-d6321ddf296c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:39:36.988848 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539233b-d0fe-48b3-bbbb-d6321ddf296c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:39:37.008254 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-5539233b-d0fe-48b3-bbbb-d6321ddf296c None None] MFA Rules not processed for user `45d8920d9a7940abb112dbbb3d7f6e65`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:39:37.009526 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5539233b-d0fe-48b3-bbbb-d6321ddf296c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:39:37.016836 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 984/1960] 10.222.0.70 () {58 vars in 1018 bytes} [Thu Oct 27 09:39:36 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:39:37.028189 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc1ea7-34f0-4ddf-9e2a-3c793b345892 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:39:37.039398 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc1ea7-34f0-4ddf-9e2a-3c793b345892 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:39:37.042897 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fce4ff2-ee6e-4d29-ac60-dbbd33729345 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:39:37.043480 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce4ff2-ee6e-4d29-ac60-dbbd33729345 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:39:37.043768 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce4ff2-ee6e-4d29-ac60-dbbd33729345 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:39:37.044012 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce4ff2-ee6e-4d29-ac60-dbbd33729345 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:39:37.050880 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fce4ff2-ee6e-4d29-ac60-dbbd33729345 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:39:37.052290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fce4ff2-ee6e-4d29-ac60-dbbd33729345 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:39:37.060441 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 977/1961] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:39:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:39:37.100284 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0676be04-5452-434d-942e-c1d13031f9d8 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:39:37.100901 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0676be04-5452-434d-942e-c1d13031f9d8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:39:37.101342 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0676be04-5452-434d-942e-c1d13031f9d8 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:39:37.108005 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 985/1962] 10.222.0.70 () {54 vars in 961 bytes} [Thu Oct 27 09:39:37 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:39:37.109885 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b554f0-5b01-403a-8d3f-a2e2fe742003 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:39:37.110169 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b554f0-5b01-403a-8d3f-a2e2fe742003 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:39:37.110441 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b554f0-5b01-403a-8d3f-a2e2fe742003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:39:37.138592 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-50b554f0-5b01-403a-8d3f-a2e2fe742003 None None] MFA Rules not processed for user `2316b30a6e7b4175bd7b90583584eab5`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:39:37.140956 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-50b554f0-5b01-403a-8d3f-a2e2fe742003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:39:37.178630 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:39:37.185271 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17be88d-cc02-4cf5-af3a-bad9260e21e3 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:39:37.185907 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17be88d-cc02-4cf5-af3a-bad9260e21e3 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:39:37.186333 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17be88d-cc02-4cf5-af3a-bad9260e21e3 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:39:37.189010 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 986/1963] 10.222.0.70 () {54 vars in 921 bytes} [Thu Oct 27 09:39:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 09:39:37.195882 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b93eae-95b9-4bea-9996-4c22e507eec4 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:39:37.196469 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b93eae-95b9-4bea-9996-4c22e507eec4 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:39:37.196929 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b93eae-95b9-4bea-9996-4c22e507eec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:39:37.224057 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-a9b93eae-95b9-4bea-9996-4c22e507eec4 None None] MFA Rules not processed for user `dbe4a6e4b6184826b30ac4fc13ad9b1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:39:37.225313 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a9b93eae-95b9-4bea-9996-4c22e507eec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:39:37.242440 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 987/1964] 10.222.0.70 () {58 vars in 1018 bytes} [Thu Oct 27 09:39:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:39:37.249908 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 978/1965] 10.222.0.70 () {58 vars in 1058 bytes} [Thu Oct 27 09:39:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:39:37.258106 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b93eae-95b9-4bea-9996-4c22e507eec4 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:39:37.276753 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b554f0-5b01-403a-8d3f-a2e2fe742003 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:39:37.278277 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b93eae-95b9-4bea-9996-4c22e507eec4 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:39:37.302232 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348f2814-08b8-4aa0-8745-7c995e882ee4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2316b30a6e7b4175bd7b90583584eab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:39:37.302870 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f2814-08b8-4aa0-8745-7c995e882ee4 service placement] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:39:37.303153 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f2814-08b8-4aa0-8745-7c995e882ee4 service placement] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:39:37.303427 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f2814-08b8-4aa0-8745-7c995e882ee4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:39:37.306684 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b554f0-5b01-403a-8d3f-a2e2fe742003 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:39:37.308878 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-348f2814-08b8-4aa0-8745-7c995e882ee4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:39:37.312341 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592bc0d5-1ab3-4c9b-9fa0-8367e5de9b68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:39:37.313698 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592bc0d5-1ab3-4c9b-9fa0-8367e5de9b68 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:39:37.314044 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592bc0d5-1ab3-4c9b-9fa0-8367e5de9b68 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:39:37.314399 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592bc0d5-1ab3-4c9b-9fa0-8367e5de9b68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:39:37.329024 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592bc0d5-1ab3-4c9b-9fa0-8367e5de9b68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:39:37.330905 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592bc0d5-1ab3-4c9b-9fa0-8367e5de9b68 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:39:37.334981 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348f2814-08b8-4aa0-8745-7c995e882ee4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:39:37.336863 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348f2814-08b8-4aa0-8745-7c995e882ee4 service placement] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:39:37.345668 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 979/1966] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:39:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 09:39:37.354266 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 988/1967] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:39:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:39:47.339248 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=22 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:39:47.349477 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=18 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:39:51.039751 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e7b33e-42de-42bc-af49-82fefd1ac506 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:39:51.040226 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e7b33e-42de-42bc-af49-82fefd1ac506 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:39:51.040370 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e7b33e-42de-42bc-af49-82fefd1ac506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:39:51.059802 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-70e7b33e-42de-42bc-af49-82fefd1ac506 None None] MFA Rules not processed for user `f7d2c6275669404db608d714baeb0065`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:39:51.061030 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-70e7b33e-42de-42bc-af49-82fefd1ac506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:39:51.115985 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 980/1968] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:39:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:39:52.447081 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a10c7-3988-48b2-ab86-4c21d4ee4053 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:39:52.447557 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a10c7-3988-48b2-ab86-4c21d4ee4053 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:39:52.447650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a10c7-3988-48b2-ab86-4c21d4ee4053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:39:52.469318 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-ea7a10c7-3988-48b2-ab86-4c21d4ee4053 None None] MFA Rules not processed for user `b4e4e3b5ce544725b1f4a439596314a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:39:52.470669 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ea7a10c7-3988-48b2-ab86-4c21d4ee4053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:39:52.530664 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 989/1969] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:39:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:39:54.913985 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07226d8-f7db-46ae-b6f6-78d33b339ffc None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:39:54.914497 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07226d8-f7db-46ae-b6f6-78d33b339ffc None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:39:54.915858 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07226d8-f7db-46ae-b6f6-78d33b339ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:39:54.958737 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-f07226d8-f7db-46ae-b6f6-78d33b339ffc None None] MFA Rules not processed for user `7f389510a6a044d8a7487561b169544b`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:39:54.961014 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f07226d8-f7db-46ae-b6f6-78d33b339ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:39:54.972373 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 981/1970] 10.222.0.70 () {58 vars in 1009 bytes} [Thu Oct 27 09:39:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:39:54.986551 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a10c7-3988-48b2-ab86-4c21d4ee4053 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:39:55.000273 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a10c7-3988-48b2-ab86-4c21d4ee4053 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:39:55.006992 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8368320-8445-4f1d-8465-37c1e0330e8d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eecf79ea4700496c89b1a67c614b34d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:39:55.007887 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8368320-8445-4f1d-8465-37c1e0330e8d service swift] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:39:55.008358 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8368320-8445-4f1d-8465-37c1e0330e8d service swift] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:39:55.008830 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8368320-8445-4f1d-8465-37c1e0330e8d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:39:55.019430 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8368320-8445-4f1d-8465-37c1e0330e8d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:39:55.021097 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8368320-8445-4f1d-8465-37c1e0330e8d service swift] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:39:55.031306 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 990/1971] 10.222.0.70 () {60 vars in 1384 bytes} [Thu Oct 27 09:39:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 27 09:40:04.949789 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=28 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:40:05.026776 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=31 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:40:10.158894 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07226d8-f7db-46ae-b6f6-78d33b339ffc None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:40:10.168042 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07226d8-f7db-46ae-b6f6-78d33b339ffc None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:40:10.172187 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a939f40-17b7-4118-809d-cc0d151786cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:40:10.172823 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a939f40-17b7-4118-809d-cc0d151786cf service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:40:10.173100 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a939f40-17b7-4118-809d-cc0d151786cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:40:10.173368 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a939f40-17b7-4118-809d-cc0d151786cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:40:10.182116 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a939f40-17b7-4118-809d-cc0d151786cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:40:10.183541 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a939f40-17b7-4118-809d-cc0d151786cf service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:40:10.193418 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 982/1972] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:40:10 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:40:17.119554 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49e735-ab76-4d00-88e3-4593a826a81a None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:40:17.120015 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49e735-ab76-4d00-88e3-4593a826a81a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:40:17.120115 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49e735-ab76-4d00-88e3-4593a826a81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:40:17.139482 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-ca49e735-ab76-4d00-88e3-4593a826a81a None None] MFA Rules not processed for user `7f389510a6a044d8a7487561b169544b`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:40:17.140712 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ca49e735-ab76-4d00-88e3-4593a826a81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:40:17.147681 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 991/1973] 10.222.0.70 () {58 vars in 967 bytes} [Thu Oct 27 09:40:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:40:17.160657 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a939f40-17b7-4118-809d-cc0d151786cf service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:40:17.172643 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a939f40-17b7-4118-809d-cc0d151786cf service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:40:17.180189 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd458655-2f4a-4dab-978a-aeacc15cac3f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eecf79ea4700496c89b1a67c614b34d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:40:17.180818 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd458655-2f4a-4dab-978a-aeacc15cac3f service swift] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:40:17.181091 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd458655-2f4a-4dab-978a-aeacc15cac3f service swift] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:40:17.181355 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd458655-2f4a-4dab-978a-aeacc15cac3f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:40:17.189625 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd458655-2f4a-4dab-978a-aeacc15cac3f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:40:17.191233 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd458655-2f4a-4dab-978a-aeacc15cac3f service swift] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:40:17.199290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 983/1974] 10.222.0.70 () {60 vars in 1384 bytes} [Thu Oct 27 09:40:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 27 09:40:27.134056 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:40:27.196029 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=12 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:41:11.133973 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca49e735-ab76-4d00-88e3-4593a826a81a None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:41:11.135446 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ca49e735-ab76-4d00-88e3-4593a826a81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:41:11.205483 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca49e735-ab76-4d00-88e3-4593a826a81a None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:41:11.211459 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660f651d-a562-4bbf-ac50-54cfb85e93bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:41:11.212107 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f651d-a562-4bbf-ac50-54cfb85e93bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:41:11.212420 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f651d-a562-4bbf-ac50-54cfb85e93bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:41:11.212726 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f651d-a562-4bbf-ac50-54cfb85e93bc admin admin] PATH_INFO: `/v3/users/b4e4e3b5ce544725b1f4a439596314a8` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:41:11.214452 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660f651d-a562-4bbf-ac50-54cfb85e93bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4e4e3b5ce544725b1f4a439596314a8)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:41:11.216421 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660f651d-a562-4bbf-ac50-54cfb85e93bc admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:41:11.273666 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-660f651d-a562-4bbf-ac50-54cfb85e93bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4e4e3b5ce544725b1f4a439596314a8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:41:11.287594 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-660f651d-a562-4bbf-ac50-54cfb85e93bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4e4e3b5ce544725b1f4a439596314a8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:41:11.301485 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-660f651d-a562-4bbf-ac50-54cfb85e93bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4e4e3b5ce544725b1f4a439596314a8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:41:11.310629 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-660f651d-a562-4bbf-ac50-54cfb85e93bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4e4e3b5ce544725b1f4a439596314a8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:41:11.311253 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-660f651d-a562-4bbf-ac50-54cfb85e93bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4e4e3b5ce544725b1f4a439596314a8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:41:11.317468 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 992/1975] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:41:11 2022] DELETE /identity/v3/users/b4e4e3b5ce544725b1f4a439596314a8 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:41:11.494531 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd458655-2f4a-4dab-978a-aeacc15cac3f service swift] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:41:11.506426 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd458655-2f4a-4dab-978a-aeacc15cac3f service swift] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:41:11.563252 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa541057-b45c-4721-81d3-ad4f93283f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:41:11.563876 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa541057-b45c-4721-81d3-ad4f93283f8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:41:11.564150 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa541057-b45c-4721-81d3-ad4f93283f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:41:11.564422 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa541057-b45c-4721-81d3-ad4f93283f8f admin admin] PATH_INFO: `/v3/projects/9dc4fbf213f04cd7ab49abad693a1cf2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:41:11.565542 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa541057-b45c-4721-81d3-ad4f93283f8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9dc4fbf213f04cd7ab49abad693a1cf2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:41:11.567434 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa541057-b45c-4721-81d3-ad4f93283f8f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:41:11.612845 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-aa541057-b45c-4721-81d3-ad4f93283f8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9dc4fbf213f04cd7ab49abad693a1cf2', 'request_id': 'req-aa541057-b45c-4721-81d3-ad4f93283f8f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:41:11.618721 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-aa541057-b45c-4721-81d3-ad4f93283f8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9dc4fbf213f04cd7ab49abad693a1cf2', 'request_id': 'req-aa541057-b45c-4721-81d3-ad4f93283f8f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:41:11.624793 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-aa541057-b45c-4721-81d3-ad4f93283f8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9dc4fbf213f04cd7ab49abad693a1cf2', 'request_id': 'req-aa541057-b45c-4721-81d3-ad4f93283f8f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:41:11.636729 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-aa541057-b45c-4721-81d3-ad4f93283f8f admin admin] The token cache is being invalidate because project 9dc4fbf213f04cd7ab49abad693a1cf2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:41:11.637010 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-aa541057-b45c-4721-81d3-ad4f93283f8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:41:11.637621 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-aa541057-b45c-4721-81d3-ad4f93283f8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:41:11.639069 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 984/1976] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:41:11 2022] DELETE /identity/v3/projects/9dc4fbf213f04cd7ab49abad693a1cf2 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:41:19.617529 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da297f9d-a3c3-4cd9-99cd-2c431b727ec7 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:41:19.618023 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da297f9d-a3c3-4cd9-99cd-2c431b727ec7 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:41:19.618142 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da297f9d-a3c3-4cd9-99cd-2c431b727ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:41:19.636965 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-da297f9d-a3c3-4cd9-99cd-2c431b727ec7 None None] MFA Rules not processed for user `2dea51c29cbd46e8bd1454e10eaac012`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:41:19.638156 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-da297f9d-a3c3-4cd9-99cd-2c431b727ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:41:19.693797 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 993/1977] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:41:19 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:41:21.633589 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:41:27.640177 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa541057-b45c-4721-81d3-ad4f93283f8f admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:41:27.641500 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-aa541057-b45c-4721-81d3-ad4f93283f8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:41:27.700235 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa541057-b45c-4721-81d3-ad4f93283f8f admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:41:27.704122 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d457959-ba4c-4792-b21c-e4c91e5a9b3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:41:27.704752 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d457959-ba4c-4792-b21c-e4c91e5a9b3b service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:41:27.705032 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d457959-ba4c-4792-b21c-e4c91e5a9b3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:41:27.705301 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d457959-ba4c-4792-b21c-e4c91e5a9b3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:41:27.706995 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1d457959-ba4c-4792-b21c-e4c91e5a9b3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:41:27.777091 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d457959-ba4c-4792-b21c-e4c91e5a9b3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:41:27.778508 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d457959-ba4c-4792-b21c-e4c91e5a9b3b service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:41:27.789133 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 985/1978] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:41:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:41:29.204757 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e1ee5d-36ba-494c-9cbf-a6048d42f605 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:41:29.205265 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e1ee5d-36ba-494c-9cbf-a6048d42f605 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:41:29.205429 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e1ee5d-36ba-494c-9cbf-a6048d42f605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:41:29.226040 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-02e1ee5d-36ba-494c-9cbf-a6048d42f605 None None] MFA Rules not processed for user `20c65ed67cd8403489949c4dea572d01`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:41:29.227226 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-02e1ee5d-36ba-494c-9cbf-a6048d42f605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:41:29.246887 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:41:29.292683 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 994/1979] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:41:29 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:41:30.413615 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d457959-ba4c-4792-b21c-e4c91e5a9b3b service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:41:30.421989 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d457959-ba4c-4792-b21c-e4c91e5a9b3b service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:41:30.425587 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d902e966-1e2b-4eaf-80bd-d858d1cac9d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:41:30.426180 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902e966-1e2b-4eaf-80bd-d858d1cac9d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:41:30.426454 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902e966-1e2b-4eaf-80bd-d858d1cac9d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:41:30.426721 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902e966-1e2b-4eaf-80bd-d858d1cac9d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:41:30.428389 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d902e966-1e2b-4eaf-80bd-d858d1cac9d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:41:30.441391 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d902e966-1e2b-4eaf-80bd-d858d1cac9d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:41:30.442800 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d902e966-1e2b-4eaf-80bd-d858d1cac9d9 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:41:30.454402 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 986/1980] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:41:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:41:39.285709 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=14 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:41:40.449400 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=60 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:41:40.698023 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeedd93-c76e-4981-9cb7-af453efcc6f0 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:41:40.698544 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeedd93-c76e-4981-9cb7-af453efcc6f0 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:41:40.698644 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeedd93-c76e-4981-9cb7-af453efcc6f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:41:40.719303 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-aeeedd93-c76e-4981-9cb7-af453efcc6f0 None None] MFA Rules not processed for user `f7d2c6275669404db608d714baeb0065`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:41:40.720509 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-aeeedd93-c76e-4981-9cb7-af453efcc6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:41:40.785646 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 995/1981] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:41:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:41:40.927846 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d0201-19c0-47b3-841b-0e60b994fbb0 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:41:40.928391 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d0201-19c0-47b3-841b-0e60b994fbb0 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:41:40.928706 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d0201-19c0-47b3-841b-0e60b994fbb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:41:40.958583 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-522d0201-19c0-47b3-841b-0e60b994fbb0 None None] MFA Rules not processed for user `286df4c587d7408d988c4739f4ff923a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:41:40.959879 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-522d0201-19c0-47b3-841b-0e60b994fbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:41:41.026550 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 987/1982] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:41:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:41:41.038222 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeedd93-c76e-4981-9cb7-af453efcc6f0 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:41:41.040466 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-aeeedd93-c76e-4981-9cb7-af453efcc6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:41:41.061582 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeedd93-c76e-4981-9cb7-af453efcc6f0 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:41:41.069444 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df99495e-c2a0-4b1c-b337-86e14bbb6cc4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe4a6e4b6184826b30ac4fc13ad9b1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:41:41.070679 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df99495e-c2a0-4b1c-b337-86e14bbb6cc4 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:41:41.071139 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df99495e-c2a0-4b1c-b337-86e14bbb6cc4 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:41:41.072080 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df99495e-c2a0-4b1c-b337-86e14bbb6cc4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:41:41.074714 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-df99495e-c2a0-4b1c-b337-86e14bbb6cc4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:41:41.086805 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df99495e-c2a0-4b1c-b337-86e14bbb6cc4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:41:41.088729 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df99495e-c2a0-4b1c-b337-86e14bbb6cc4 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:41:41.100559 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 996/1983] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:41:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:41:51.018832 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=25 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:41:51.095213 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=31 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:42:08.529341 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d0201-19c0-47b3-841b-0e60b994fbb0 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:08.538924 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d0201-19c0-47b3-841b-0e60b994fbb0 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:08.543566 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59320246-8d46-4aeb-89db-5ea4bcd5e8a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:08.544198 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59320246-8d46-4aeb-89db-5ea4bcd5e8a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:08.544484 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59320246-8d46-4aeb-89db-5ea4bcd5e8a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:08.544788 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59320246-8d46-4aeb-89db-5ea4bcd5e8a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:08.546567 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-59320246-8d46-4aeb-89db-5ea4bcd5e8a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:08.555863 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59320246-8d46-4aeb-89db-5ea4bcd5e8a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:08.557941 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59320246-8d46-4aeb-89db-5ea4bcd5e8a6 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:08.566026 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 988/1984] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:42:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:42:12.632627 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df99495e-c2a0-4b1c-b337-86e14bbb6cc4 service nova] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:12.643343 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df99495e-c2a0-4b1c-b337-86e14bbb6cc4 service nova] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:12.649079 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb94bc5-563e-4c02-8e1b-2f276a32208b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe4a6e4b6184826b30ac4fc13ad9b1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:12.650254 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb94bc5-563e-4c02-8e1b-2f276a32208b service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:12.650532 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb94bc5-563e-4c02-8e1b-2f276a32208b service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:12.650809 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb94bc5-563e-4c02-8e1b-2f276a32208b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:12.653594 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5fb94bc5-563e-4c02-8e1b-2f276a32208b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:12.666740 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb94bc5-563e-4c02-8e1b-2f276a32208b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:12.668316 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb94bc5-563e-4c02-8e1b-2f276a32208b service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:12.679172 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 997/1985] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:42:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:42:13.553987 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59320246-8d46-4aeb-89db-5ea4bcd5e8a6 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:13.555541 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-59320246-8d46-4aeb-89db-5ea4bcd5e8a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:13.631184 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59320246-8d46-4aeb-89db-5ea4bcd5e8a6 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:13.636862 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e45515-bdc9-4b7a-916a-cfc4525b21a4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2316b30a6e7b4175bd7b90583584eab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:13.637471 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e45515-bdc9-4b7a-916a-cfc4525b21a4 service placement] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:13.637751 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e45515-bdc9-4b7a-916a-cfc4525b21a4 service placement] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:13.638018 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e45515-bdc9-4b7a-916a-cfc4525b21a4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:13.639712 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-39e45515-bdc9-4b7a-916a-cfc4525b21a4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:13.652454 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e45515-bdc9-4b7a-916a-cfc4525b21a4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:13.654078 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e45515-bdc9-4b7a-916a-cfc4525b21a4 service placement] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:13.664165 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 989/1986] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:42:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:42:17.898096 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb94bc5-563e-4c02-8e1b-2f276a32208b service nova] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:17.900151 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5fb94bc5-563e-4c02-8e1b-2f276a32208b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:17.964018 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb94bc5-563e-4c02-8e1b-2f276a32208b service nova] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:17.967759 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5fbbc8-2d0c-4871-8872-045ff5683377 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:17.968367 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5fbbc8-2d0c-4871-8872-045ff5683377 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:17.968704 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5fbbc8-2d0c-4871-8872-045ff5683377 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:17.968984 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5fbbc8-2d0c-4871-8872-045ff5683377 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:17.970696 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2a5fbbc8-2d0c-4871-8872-045ff5683377 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:17.982334 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5fbbc8-2d0c-4871-8872-045ff5683377 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:17.983753 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5fbbc8-2d0c-4871-8872-045ff5683377 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:17.992703 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 998/1987] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:42:17 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:42:19.670661 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66476c5-8366-4c5c-8453-f504412da813 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:19.671475 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66476c5-8366-4c5c-8453-f504412da813 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:19.671754 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66476c5-8366-4c5c-8453-f504412da813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:19.693830 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b66476c5-8366-4c5c-8453-f504412da813 None None] MFA Rules not processed for user `7f389510a6a044d8a7487561b169544b`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:42:19.695169 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b66476c5-8366-4c5c-8453-f504412da813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:19.757333 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 990/1988] 10.222.0.70 () {58 vars in 967 bytes} [Thu Oct 27 09:42:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:42:19.818888 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5fbbc8-2d0c-4871-8872-045ff5683377 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:19.820400 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2a5fbbc8-2d0c-4871-8872-045ff5683377 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:19.888135 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5fbbc8-2d0c-4871-8872-045ff5683377 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:19.893084 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e63ac87-7a11-415b-bd0e-1177fccc8884 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eecf79ea4700496c89b1a67c614b34d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:19.893687 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63ac87-7a11-415b-bd0e-1177fccc8884 service swift] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:19.893966 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63ac87-7a11-415b-bd0e-1177fccc8884 service swift] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:19.894231 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63ac87-7a11-415b-bd0e-1177fccc8884 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:19.902045 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e63ac87-7a11-415b-bd0e-1177fccc8884 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:19.903615 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e63ac87-7a11-415b-bd0e-1177fccc8884 service swift] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:19.911648 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 999/1989] 10.222.0.70 () {60 vars in 1384 bytes} [Thu Oct 27 09:42:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 27 09:42:22.842774 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66476c5-8366-4c5c-8453-f504412da813 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:22.844189 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b66476c5-8366-4c5c-8453-f504412da813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:22.860624 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66476c5-8366-4c5c-8453-f504412da813 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:22.866210 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c37372c-e556-4532-842a-9ecefd1d4f17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe4a6e4b6184826b30ac4fc13ad9b1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:22.866819 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c37372c-e556-4532-842a-9ecefd1d4f17 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:22.867127 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c37372c-e556-4532-842a-9ecefd1d4f17 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:22.867397 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c37372c-e556-4532-842a-9ecefd1d4f17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:22.869758 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6c37372c-e556-4532-842a-9ecefd1d4f17 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:22.880277 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c37372c-e556-4532-842a-9ecefd1d4f17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:22.881975 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c37372c-e556-4532-842a-9ecefd1d4f17 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:22.890909 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 991/1990] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:42:22 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:42:23.849848 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e63ac87-7a11-415b-bd0e-1177fccc8884 service swift] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:23.851336 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9e63ac87-7a11-415b-bd0e-1177fccc8884 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:23.866696 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e63ac87-7a11-415b-bd0e-1177fccc8884 service swift] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:23.872831 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0c3950-bd66-487f-b524-8f433b271b62 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2316b30a6e7b4175bd7b90583584eab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:23.873550 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c3950-bd66-487f-b524-8f433b271b62 service placement] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:23.873922 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c3950-bd66-487f-b524-8f433b271b62 service placement] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:23.874294 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c3950-bd66-487f-b524-8f433b271b62 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:23.876844 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5f0c3950-bd66-487f-b524-8f433b271b62 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:23.891235 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0c3950-bd66-487f-b524-8f433b271b62 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:23.893255 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0c3950-bd66-487f-b524-8f433b271b62 service placement] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:23.906142 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1000/1991] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:42:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 09:42:28.759815 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c37372c-e556-4532-842a-9ecefd1d4f17 service nova] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:28.771010 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c37372c-e556-4532-842a-9ecefd1d4f17 service nova] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:28.776751 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ab77b-a11b-412e-af37-3d48233bfe57 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2316b30a6e7b4175bd7b90583584eab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:28.777534 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ab77b-a11b-412e-af37-3d48233bfe57 service placement] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:28.777858 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ab77b-a11b-412e-af37-3d48233bfe57 service placement] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:28.778308 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ab77b-a11b-412e-af37-3d48233bfe57 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:28.780237 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8f5ab77b-a11b-412e-af37-3d48233bfe57 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:28.793637 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5ab77b-a11b-412e-af37-3d48233bfe57 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:28.795314 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5ab77b-a11b-412e-af37-3d48233bfe57 service placement] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:28.808865 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 992/1992] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:42:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:42:28.824401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0c3950-bd66-487f-b524-8f433b271b62 service placement] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:28.826296 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5f0c3950-bd66-487f-b524-8f433b271b62 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:28.903530 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0c3950-bd66-487f-b524-8f433b271b62 service placement] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:28.907936 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30bd18e-7d6f-4824-bb71-a923d53fc8a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:28.908602 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30bd18e-7d6f-4824-bb71-a923d53fc8a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:28.908886 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30bd18e-7d6f-4824-bb71-a923d53fc8a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:28.909158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30bd18e-7d6f-4824-bb71-a923d53fc8a7 admin admin] PATH_INFO: `/v3/users/51aba36d56c94e2fa351aa098dc0a962` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:28.910267 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30bd18e-7d6f-4824-bb71-a923d53fc8a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51aba36d56c94e2fa351aa098dc0a962)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:28.912171 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30bd18e-7d6f-4824-bb71-a923d53fc8a7 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:28.917529 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:42:28.958514 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d30bd18e-7d6f-4824-bb71-a923d53fc8a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51aba36d56c94e2fa351aa098dc0a962'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:28.966002 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d30bd18e-7d6f-4824-bb71-a923d53fc8a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51aba36d56c94e2fa351aa098dc0a962'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:28.977788 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d30bd18e-7d6f-4824-bb71-a923d53fc8a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51aba36d56c94e2fa351aa098dc0a962'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:28.985200 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d30bd18e-7d6f-4824-bb71-a923d53fc8a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51aba36d56c94e2fa351aa098dc0a962'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:28.985822 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d30bd18e-7d6f-4824-bb71-a923d53fc8a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51aba36d56c94e2fa351aa098dc0a962'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:28.991784 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1001/1993] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:42:28 2022] DELETE /identity/v3/users/51aba36d56c94e2fa351aa098dc0a962 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:42:29.163937 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ab77b-a11b-412e-af37-3d48233bfe57 service placement] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:29.174508 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ab77b-a11b-412e-af37-3d48233bfe57 service placement] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:29.226431 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:42:29.233462 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963ce727-68d5-4f29-953d-2889f71fc74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:29.234093 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963ce727-68d5-4f29-953d-2889f71fc74b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:29.234419 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963ce727-68d5-4f29-953d-2889f71fc74b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:29.234691 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963ce727-68d5-4f29-953d-2889f71fc74b admin admin] PATH_INFO: `/v3/projects/f169a47a4c9a4c60b87cedcd77c102c8` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:29.235834 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963ce727-68d5-4f29-953d-2889f71fc74b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f169a47a4c9a4c60b87cedcd77c102c8)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:29.237827 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963ce727-68d5-4f29-953d-2889f71fc74b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:29.283351 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-963ce727-68d5-4f29-953d-2889f71fc74b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f169a47a4c9a4c60b87cedcd77c102c8', 'request_id': 'req-963ce727-68d5-4f29-953d-2889f71fc74b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:29.289668 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-963ce727-68d5-4f29-953d-2889f71fc74b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f169a47a4c9a4c60b87cedcd77c102c8', 'request_id': 'req-963ce727-68d5-4f29-953d-2889f71fc74b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:29.295290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-963ce727-68d5-4f29-953d-2889f71fc74b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f169a47a4c9a4c60b87cedcd77c102c8', 'request_id': 'req-963ce727-68d5-4f29-953d-2889f71fc74b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:29.308621 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-963ce727-68d5-4f29-953d-2889f71fc74b admin admin] The token cache is being invalidate because project f169a47a4c9a4c60b87cedcd77c102c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:42:29.308931 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-963ce727-68d5-4f29-953d-2889f71fc74b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:29.309538 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-963ce727-68d5-4f29-953d-2889f71fc74b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:29.314177 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 993/1994] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:42:29 2022] DELETE /identity/v3/projects/f169a47a4c9a4c60b87cedcd77c102c8 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:42:29.315219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30bd18e-7d6f-4824-bb71-a923d53fc8a7 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:29.316717 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d30bd18e-7d6f-4824-bb71-a923d53fc8a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:29.330691 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30bd18e-7d6f-4824-bb71-a923d53fc8a7 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:29.393142 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac836e3-bd1c-4225-b4d9-e89c806be048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:29.393788 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac836e3-bd1c-4225-b4d9-e89c806be048 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:29.394070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac836e3-bd1c-4225-b4d9-e89c806be048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:29.394345 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac836e3-bd1c-4225-b4d9-e89c806be048 admin admin] PATH_INFO: `/v3/users/81100e3911bc4ac3a819e562d3dc97e0` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:29.396439 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac836e3-bd1c-4225-b4d9-e89c806be048 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81100e3911bc4ac3a819e562d3dc97e0)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:29.399343 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac836e3-bd1c-4225-b4d9-e89c806be048 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:29.457636 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-1ac836e3-bd1c-4225-b4d9-e89c806be048 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81100e3911bc4ac3a819e562d3dc97e0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:29.466947 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-1ac836e3-bd1c-4225-b4d9-e89c806be048 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81100e3911bc4ac3a819e562d3dc97e0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:29.480842 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-1ac836e3-bd1c-4225-b4d9-e89c806be048 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81100e3911bc4ac3a819e562d3dc97e0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:29.489726 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-1ac836e3-bd1c-4225-b4d9-e89c806be048 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81100e3911bc4ac3a819e562d3dc97e0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:29.490356 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-1ac836e3-bd1c-4225-b4d9-e89c806be048 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81100e3911bc4ac3a819e562d3dc97e0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:29.496968 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1002/1995] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:42:29 2022] DELETE /identity/v3/users/81100e3911bc4ac3a819e562d3dc97e0 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:42:29.541887 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963ce727-68d5-4f29-953d-2889f71fc74b admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:29.546534 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-963ce727-68d5-4f29-953d-2889f71fc74b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:29.630600 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963ce727-68d5-4f29-953d-2889f71fc74b admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:29.679908 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac836e3-bd1c-4225-b4d9-e89c806be048 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:29.683374 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1ac836e3-bd1c-4225-b4d9-e89c806be048 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:29.692354 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:42:29.712786 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954665db-7ed1-4595-86db-4475c0d509ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe4a6e4b6184826b30ac4fc13ad9b1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:29.713413 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954665db-7ed1-4595-86db-4475c0d509ce service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:29.713688 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954665db-7ed1-4595-86db-4475c0d509ce service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:29.713955 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954665db-7ed1-4595-86db-4475c0d509ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:29.715821 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-954665db-7ed1-4595-86db-4475c0d509ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:29.729329 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954665db-7ed1-4595-86db-4475c0d509ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:29.735460 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954665db-7ed1-4595-86db-4475c0d509ce service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:29.759813 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 994/1996] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:42:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:42:29.796413 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac836e3-bd1c-4225-b4d9-e89c806be048 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:29.811274 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b225738-574d-4a5e-9a97-b9b1d6e530ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:29.811950 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b225738-574d-4a5e-9a97-b9b1d6e530ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:29.812235 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b225738-574d-4a5e-9a97-b9b1d6e530ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:29.812590 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b225738-574d-4a5e-9a97-b9b1d6e530ee admin admin] PATH_INFO: `/v3/projects/39b5b195f9b04586b6e8e4a33271619c` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:29.813775 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b225738-574d-4a5e-9a97-b9b1d6e530ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=39b5b195f9b04586b6e8e4a33271619c)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:29.815909 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b225738-574d-4a5e-9a97-b9b1d6e530ee admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:29.875446 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8b225738-574d-4a5e-9a97-b9b1d6e530ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39b5b195f9b04586b6e8e4a33271619c', 'request_id': 'req-8b225738-574d-4a5e-9a97-b9b1d6e530ee'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:29.882642 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8b225738-574d-4a5e-9a97-b9b1d6e530ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39b5b195f9b04586b6e8e4a33271619c', 'request_id': 'req-8b225738-574d-4a5e-9a97-b9b1d6e530ee'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:29.896788 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8b225738-574d-4a5e-9a97-b9b1d6e530ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39b5b195f9b04586b6e8e4a33271619c', 'request_id': 'req-8b225738-574d-4a5e-9a97-b9b1d6e530ee'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:29.912956 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8b225738-574d-4a5e-9a97-b9b1d6e530ee admin admin] The token cache is being invalidate because project 39b5b195f9b04586b6e8e4a33271619c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:42:29.913331 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8b225738-574d-4a5e-9a97-b9b1d6e530ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:29.914078 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8b225738-574d-4a5e-9a97-b9b1d6e530ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:29.920609 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1003/1997] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:42:29 2022] DELETE /identity/v3/projects/39b5b195f9b04586b6e8e4a33271619c => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:42:38.584188 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954665db-7ed1-4595-86db-4475c0d509ce service nova] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:38.585642 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-954665db-7ed1-4595-86db-4475c0d509ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:38.596220 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:42:38.647550 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954665db-7ed1-4595-86db-4475c0d509ce service nova] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:38.699390 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6541311-292e-4b47-a66b-c44d83215bfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:38.700269 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6541311-292e-4b47-a66b-c44d83215bfd service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:38.700570 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6541311-292e-4b47-a66b-c44d83215bfd service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:38.700843 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6541311-292e-4b47-a66b-c44d83215bfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:38.702554 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f6541311-292e-4b47-a66b-c44d83215bfd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:38.764801 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6541311-292e-4b47-a66b-c44d83215bfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:38.766365 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6541311-292e-4b47-a66b-c44d83215bfd service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:38.776009 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 995/1998] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:42:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:42:39.907918 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=49,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:42:42.334245 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b225738-574d-4a5e-9a97-b9b1d6e530ee admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:42.336124 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8b225738-574d-4a5e-9a97-b9b1d6e530ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:42.349561 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b225738-574d-4a5e-9a97-b9b1d6e530ee admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:42.354013 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36d4b1-fac5-4566-af3e-6a312ca835af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:42.354641 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36d4b1-fac5-4566-af3e-6a312ca835af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:42.354914 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36d4b1-fac5-4566-af3e-6a312ca835af admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:42.355179 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36d4b1-fac5-4566-af3e-6a312ca835af admin admin] PATH_INFO: `/v3/users/20c65ed67cd8403489949c4dea572d01` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:42.356294 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e36d4b1-fac5-4566-af3e-6a312ca835af admin admin] RBAC: Authorizing `identity:delete_user(user_id=20c65ed67cd8403489949c4dea572d01)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:42.358247 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e36d4b1-fac5-4566-af3e-6a312ca835af admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:42.415936 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6e36d4b1-fac5-4566-af3e-6a312ca835af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20c65ed67cd8403489949c4dea572d01'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:42.422885 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6e36d4b1-fac5-4566-af3e-6a312ca835af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20c65ed67cd8403489949c4dea572d01'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:42.434868 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6e36d4b1-fac5-4566-af3e-6a312ca835af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20c65ed67cd8403489949c4dea572d01'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:42.445015 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6e36d4b1-fac5-4566-af3e-6a312ca835af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20c65ed67cd8403489949c4dea572d01'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:42.445770 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6e36d4b1-fac5-4566-af3e-6a312ca835af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20c65ed67cd8403489949c4dea572d01'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:42.455602 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1004/1999] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:42:42 2022] DELETE /identity/v3/users/20c65ed67cd8403489949c4dea572d01 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:42:42.652442 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6541311-292e-4b47-a66b-c44d83215bfd service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:42.667560 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6541311-292e-4b47-a66b-c44d83215bfd service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:42.726833 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5cf0a8-4bfe-4112-83e9-36043afa792c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:42.727613 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5cf0a8-4bfe-4112-83e9-36043afa792c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:42.727893 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5cf0a8-4bfe-4112-83e9-36043afa792c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:42.728827 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5cf0a8-4bfe-4112-83e9-36043afa792c admin admin] PATH_INFO: `/v3/projects/df306d37ff634f7eaad80fc3bc65b653` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:42.730107 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5cf0a8-4bfe-4112-83e9-36043afa792c admin admin] RBAC: Authorizing `identity:delete_project(project_id=df306d37ff634f7eaad80fc3bc65b653)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:42.732081 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5cf0a8-4bfe-4112-83e9-36043afa792c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:42.749205 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:42:42.782728 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6a5cf0a8-4bfe-4112-83e9-36043afa792c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df306d37ff634f7eaad80fc3bc65b653', 'request_id': 'req-6a5cf0a8-4bfe-4112-83e9-36043afa792c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:42.788829 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6a5cf0a8-4bfe-4112-83e9-36043afa792c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df306d37ff634f7eaad80fc3bc65b653', 'request_id': 'req-6a5cf0a8-4bfe-4112-83e9-36043afa792c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:42.794840 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6a5cf0a8-4bfe-4112-83e9-36043afa792c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df306d37ff634f7eaad80fc3bc65b653', 'request_id': 'req-6a5cf0a8-4bfe-4112-83e9-36043afa792c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:42.808054 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6a5cf0a8-4bfe-4112-83e9-36043afa792c admin admin] The token cache is being invalidate because project df306d37ff634f7eaad80fc3bc65b653 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:42:42.808299 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6a5cf0a8-4bfe-4112-83e9-36043afa792c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:42.809076 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6a5cf0a8-4bfe-4112-83e9-36043afa792c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:42.810753 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 996/2000] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:42:42 2022] DELETE /identity/v3/projects/df306d37ff634f7eaad80fc3bc65b653 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:42:42.815809 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a401484-170a-40a2-99ce-08c20051eeb8 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:42.816176 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a401484-170a-40a2-99ce-08c20051eeb8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:42.816465 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a401484-170a-40a2-99ce-08c20051eeb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:42.846858 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-9a401484-170a-40a2-99ce-08c20051eeb8 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:42:42.848720 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9a401484-170a-40a2-99ce-08c20051eeb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:42.913784 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1005/2001] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:42:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:42:42.921717 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9245b62-7950-4ba4-a7e8-23e6652915fb None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:42.921929 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9245b62-7950-4ba4-a7e8-23e6652915fb None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:42.922124 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9245b62-7950-4ba4-a7e8-23e6652915fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:42.943129 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-e9245b62-7950-4ba4-a7e8-23e6652915fb None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:42:42.944370 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e9245b62-7950-4ba4-a7e8-23e6652915fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:42.953845 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 997/2002] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:42:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:42:42.957876 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a401484-170a-40a2-99ce-08c20051eeb8 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:42.967999 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a401484-170a-40a2-99ce-08c20051eeb8 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:42.972502 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b7d9f-4e2e-47fe-897c-0b2247eb56c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:42.973127 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b7d9f-4e2e-47fe-897c-0b2247eb56c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:42.973416 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b7d9f-4e2e-47fe-897c-0b2247eb56c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:42.974600 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b7d9f-4e2e-47fe-897c-0b2247eb56c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:42.975240 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846b7d9f-4e2e-47fe-897c-0b2247eb56c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:42:42.975537 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846b7d9f-4e2e-47fe-897c-0b2247eb56c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:42.977494 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846b7d9f-4e2e-47fe-897c-0b2247eb56c3 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:42.989796 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1006/2003] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:42:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:42:42.994681 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9245b62-7950-4ba4-a7e8-23e6652915fb None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:43.003841 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9245b62-7950-4ba4-a7e8-23e6652915fb None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:43.008530 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15646f7d-b8a5-4fa1-8cd8-963ffd9a604a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:43.009136 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15646f7d-b8a5-4fa1-8cd8-963ffd9a604a admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:43.009416 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15646f7d-b8a5-4fa1-8cd8-963ffd9a604a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:43.009685 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15646f7d-b8a5-4fa1-8cd8-963ffd9a604a admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:43.010313 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15646f7d-b8a5-4fa1-8cd8-963ffd9a604a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:43.012208 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15646f7d-b8a5-4fa1-8cd8-963ffd9a604a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:43.038437 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 998/2004] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:42:42 2022] POST /identity/v3/projects => generated 367 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:42:43.043011 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b7d9f-4e2e-47fe-897c-0b2247eb56c3 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:43.052934 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b7d9f-4e2e-47fe-897c-0b2247eb56c3 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:43.108632 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95435721-0f05-4c0a-b66a-6059c5bcbc50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:43.109238 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95435721-0f05-4c0a-b66a-6059c5bcbc50 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:43.109515 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95435721-0f05-4c0a-b66a-6059c5bcbc50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:43.109782 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95435721-0f05-4c0a-b66a-6059c5bcbc50 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:43.110508 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95435721-0f05-4c0a-b66a-6059c5bcbc50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:43.112585 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95435721-0f05-4c0a-b66a-6059c5bcbc50 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:43.121182 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:42:43.135828 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1007/2005] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:42:43 2022] POST /identity/v3/users => generated 284 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:42:43.139992 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15646f7d-b8a5-4fa1-8cd8-963ffd9a604a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:43.150333 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15646f7d-b8a5-4fa1-8cd8-963ffd9a604a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:43.154927 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7f86d7-114f-48b9-8c15-0a9abb106bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:43.155519 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7f86d7-114f-48b9-8c15-0a9abb106bb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:43.155796 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7f86d7-114f-48b9-8c15-0a9abb106bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:43.156067 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7f86d7-114f-48b9-8c15-0a9abb106bb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:43.156739 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7f86d7-114f-48b9-8c15-0a9abb106bb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:43.158330 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7f86d7-114f-48b9-8c15-0a9abb106bb2 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:43.168681 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 999/2006] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:42:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:42:43.172350 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95435721-0f05-4c0a-b66a-6059c5bcbc50 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:43.183092 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95435721-0f05-4c0a-b66a-6059c5bcbc50 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:43.187665 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da3773d-753b-424b-ba5c-2e6f53bc72a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:43.188253 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da3773d-753b-424b-ba5c-2e6f53bc72a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:43.188544 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da3773d-753b-424b-ba5c-2e6f53bc72a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:43.188853 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da3773d-753b-424b-ba5c-2e6f53bc72a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:43.189470 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da3773d-753b-424b-ba5c-2e6f53bc72a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:43.191034 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da3773d-753b-424b-ba5c-2e6f53bc72a0 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:43.202615 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1008/2007] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:42:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:42:43.206769 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7f86d7-114f-48b9-8c15-0a9abb106bb2 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:43.216922 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7f86d7-114f-48b9-8c15-0a9abb106bb2 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:43.221611 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b6a2b6-4b46-43f0-b603-a749fb570433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:43.222330 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b6a2b6-4b46-43f0-b603-a749fb570433 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:43.222725 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b6a2b6-4b46-43f0-b603-a749fb570433 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:43.222942 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b6a2b6-4b46-43f0-b603-a749fb570433 admin admin] PATH_INFO: `/v3/projects/23605b91ac6e48aeac842caa26a5d2be/users/6282394da5564fe8aab58996a3d89187/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:43.238204 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b6a2b6-4b46-43f0-b603-a749fb570433 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23605b91ac6e48aeac842caa26a5d2be, user_id=6282394da5564fe8aab58996a3d89187, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:43.241034 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b6a2b6-4b46-43f0-b603-a749fb570433 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:43.254471 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1000/2008] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:42:43 2022] PUT /identity/v3/projects/23605b91ac6e48aeac842caa26a5d2be/users/6282394da5564fe8aab58996a3d89187/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:42:43.265705 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5277372-60e2-4a5b-bb82-4c66168ba474 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:43.265998 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5277372-60e2-4a5b-bb82-4c66168ba474 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:43.266270 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5277372-60e2-4a5b-bb82-4c66168ba474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:43.296448 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-b5277372-60e2-4a5b-bb82-4c66168ba474 None None] MFA Rules not processed for user `6282394da5564fe8aab58996a3d89187`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:42:43.297790 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b5277372-60e2-4a5b-bb82-4c66168ba474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:43.361537 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1009/2009] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:42:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:42:43.366078 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b6a2b6-4b46-43f0-b603-a749fb570433 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:43.428357 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b6a2b6-4b46-43f0-b603-a749fb570433 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:43.432989 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a15e5a-d081-4503-acda-a8693f3a0182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:43.433587 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a15e5a-d081-4503-acda-a8693f3a0182 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:43.433866 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a15e5a-d081-4503-acda-a8693f3a0182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:43.434140 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a15e5a-d081-4503-acda-a8693f3a0182 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:43.434767 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a15e5a-d081-4503-acda-a8693f3a0182 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:43.436696 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a15e5a-d081-4503-acda-a8693f3a0182 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:43.463304 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1001/2010] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:42:43 2022] POST /identity/v3/projects => generated 367 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:42:43.467381 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5277372-60e2-4a5b-bb82-4c66168ba474 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:43.478904 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5277372-60e2-4a5b-bb82-4c66168ba474 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:43.537393 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0952555-9e11-405b-a2a5-a32c142dc232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:43.537921 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0952555-9e11-405b-a2a5-a32c142dc232 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:43.538198 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0952555-9e11-405b-a2a5-a32c142dc232 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:43.538467 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0952555-9e11-405b-a2a5-a32c142dc232 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:43.539128 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0952555-9e11-405b-a2a5-a32c142dc232 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:43.541630 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0952555-9e11-405b-a2a5-a32c142dc232 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:43.564091 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1010/2011] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:42:43 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:42:43.568862 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a15e5a-d081-4503-acda-a8693f3a0182 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:43.579957 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a15e5a-d081-4503-acda-a8693f3a0182 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:43.584621 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3d3caa-91d3-4595-945f-879394174fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:43.585223 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3d3caa-91d3-4595-945f-879394174fe6 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:43.585501 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3d3caa-91d3-4595-945f-879394174fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:43.585771 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3d3caa-91d3-4595-945f-879394174fe6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:43.586418 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3d3caa-91d3-4595-945f-879394174fe6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:43.588590 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3d3caa-91d3-4595-945f-879394174fe6 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:43.599433 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1002/2012] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:42:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 09:42:43.604258 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0952555-9e11-405b-a2a5-a32c142dc232 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:43.614663 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0952555-9e11-405b-a2a5-a32c142dc232 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:43.619357 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c03d61d-c8df-44f8-b741-f9330d36b663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:43.620016 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03d61d-c8df-44f8-b741-f9330d36b663 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:43.620294 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03d61d-c8df-44f8-b741-f9330d36b663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:43.620591 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03d61d-c8df-44f8-b741-f9330d36b663 admin admin] PATH_INFO: `/v3/projects/f1adf863d3b6482fa6646f408a46ca19/users/c5374286686147119b9f6edde7ac556d/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:43.642219 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c03d61d-c8df-44f8-b741-f9330d36b663 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1adf863d3b6482fa6646f408a46ca19, user_id=c5374286686147119b9f6edde7ac556d, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:43.645068 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c03d61d-c8df-44f8-b741-f9330d36b663 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:43.659835 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1011/2013] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:42:43 2022] PUT /identity/v3/projects/f1adf863d3b6482fa6646f408a46ca19/users/c5374286686147119b9f6edde7ac556d/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:42:43.663782 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3d3caa-91d3-4595-945f-879394174fe6 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:43.674368 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3d3caa-91d3-4595-945f-879394174fe6 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:43.685662 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:42:43.734887 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2e1aa1-2d8c-4399-ad99-3c2100d95c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:43.735502 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e1aa1-2d8c-4399-ad99-3c2100d95c96 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:43.735778 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e1aa1-2d8c-4399-ad99-3c2100d95c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:43.736050 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e1aa1-2d8c-4399-ad99-3c2100d95c96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:43.736715 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2e1aa1-2d8c-4399-ad99-3c2100d95c96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:43.738309 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2e1aa1-2d8c-4399-ad99-3c2100d95c96 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:43.751373 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1003/2014] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:42:43 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:42:43.752880 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c03d61d-c8df-44f8-b741-f9330d36b663 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:43.763696 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c03d61d-c8df-44f8-b741-f9330d36b663 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:43.768487 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7deb16-bba7-45b2-a5ba-3fe1a2258886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:43.769197 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7deb16-bba7-45b2-a5ba-3fe1a2258886 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:43.769475 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7deb16-bba7-45b2-a5ba-3fe1a2258886 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:43.769746 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7deb16-bba7-45b2-a5ba-3fe1a2258886 admin admin] PATH_INFO: `/v3/projects/f1adf863d3b6482fa6646f408a46ca19/users/c5374286686147119b9f6edde7ac556d/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:43.771916 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7deb16-bba7-45b2-a5ba-3fe1a2258886 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1adf863d3b6482fa6646f408a46ca19, user_id=c5374286686147119b9f6edde7ac556d, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:43.774577 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7deb16-bba7-45b2-a5ba-3fe1a2258886 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:43.786214 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1012/2015] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:42:43 2022] PUT /identity/v3/projects/f1adf863d3b6482fa6646f408a46ca19/users/c5374286686147119b9f6edde7ac556d/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:42:43.796269 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a814f85-5ace-473d-b56c-1c526c44cc8a None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:43.796572 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a814f85-5ace-473d-b56c-1c526c44cc8a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:43.796843 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a814f85-5ace-473d-b56c-1c526c44cc8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:43.825946 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-5a814f85-5ace-473d-b56c-1c526c44cc8a None None] MFA Rules not processed for user `c5374286686147119b9f6edde7ac556d`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:42:43.827143 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5a814f85-5ace-473d-b56c-1c526c44cc8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:43.905452 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1004/2016] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:42:43 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:42:43.950156 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7deb16-bba7-45b2-a5ba-3fe1a2258886 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:43.950156 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ea7deb16-bba7-45b2-a5ba-3fe1a2258886 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:43.970787 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:42:44.014091 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7deb16-bba7-45b2-a5ba-3fe1a2258886 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:44.017923 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3b4b5c-c99b-48bc-8b6b-a350af41d283 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:44.018540 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b4b5c-c99b-48bc-8b6b-a350af41d283 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:44.019026 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b4b5c-c99b-48bc-8b6b-a350af41d283 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:44.019306 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b4b5c-c99b-48bc-8b6b-a350af41d283 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:44.030653 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3b4b5c-c99b-48bc-8b6b-a350af41d283 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:44.032138 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3b4b5c-c99b-48bc-8b6b-a350af41d283 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:44.043585 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1013/2017] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:42:43 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 09:42:44.094586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a814f85-5ace-473d-b56c-1c526c44cc8a None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:44.096006 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5a814f85-5ace-473d-b56c-1c526c44cc8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:44.109674 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a814f85-5ace-473d-b56c-1c526c44cc8a None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:44.113448 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7f8823-57ed-4d91-9a30-21ef18d221c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:44.114056 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7f8823-57ed-4d91-9a30-21ef18d221c4 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:44.114331 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7f8823-57ed-4d91-9a30-21ef18d221c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:44.114606 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7f8823-57ed-4d91-9a30-21ef18d221c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:44.171608 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7f8823-57ed-4d91-9a30-21ef18d221c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:44.173181 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7f8823-57ed-4d91-9a30-21ef18d221c4 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:44.183052 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1005/2018] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:42:44 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:42:47.489445 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3b4b5c-c99b-48bc-8b6b-a350af41d283 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:47.553747 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3b4b5c-c99b-48bc-8b6b-a350af41d283 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:47.558550 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1eb868-4367-479b-8539-8bfdaeadc609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:47.559167 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1eb868-4367-479b-8539-8bfdaeadc609 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:47.559441 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1eb868-4367-479b-8539-8bfdaeadc609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:47.559707 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1eb868-4367-479b-8539-8bfdaeadc609 admin admin] PATH_INFO: `/v3/users/6282394da5564fe8aab58996a3d89187` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:47.560838 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1eb868-4367-479b-8539-8bfdaeadc609 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6282394da5564fe8aab58996a3d89187)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:47.562723 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1eb868-4367-479b-8539-8bfdaeadc609 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:47.610151 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8f1eb868-4367-479b-8539-8bfdaeadc609 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6282394da5564fe8aab58996a3d89187'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:47.617274 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8f1eb868-4367-479b-8539-8bfdaeadc609 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6282394da5564fe8aab58996a3d89187'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:47.629082 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8f1eb868-4367-479b-8539-8bfdaeadc609 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6282394da5564fe8aab58996a3d89187'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:47.636354 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8f1eb868-4367-479b-8539-8bfdaeadc609 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6282394da5564fe8aab58996a3d89187'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:47.637005 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8f1eb868-4367-479b-8539-8bfdaeadc609 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6282394da5564fe8aab58996a3d89187'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:47.643250 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1014/2019] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:42:47 2022] DELETE /identity/v3/users/6282394da5564fe8aab58996a3d89187 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:42:47.647668 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80666b1e-f3e8-44cc-a8e5-012077a614e3 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:47.648133 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80666b1e-f3e8-44cc-a8e5-012077a614e3 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:47.648244 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80666b1e-f3e8-44cc-a8e5-012077a614e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:47.667775 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-80666b1e-f3e8-44cc-a8e5-012077a614e3 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:42:47.669013 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-80666b1e-f3e8-44cc-a8e5-012077a614e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:47.698918 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:42:47.727813 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1006/2020] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:42:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:42:47.737536 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1eb868-4367-479b-8539-8bfdaeadc609 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:47.738956 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8f1eb868-4367-479b-8539-8bfdaeadc609 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:47.784642 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:42:47.797733 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1eb868-4367-479b-8539-8bfdaeadc609 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:47.801407 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b1855-741b-4cd7-a2c9-20a96f7d2a41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:47.802009 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b1855-741b-4cd7-a2c9-20a96f7d2a41 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:47.802284 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b1855-741b-4cd7-a2c9-20a96f7d2a41 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:47.802552 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b1855-741b-4cd7-a2c9-20a96f7d2a41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:47.811416 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1b1855-741b-4cd7-a2c9-20a96f7d2a41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:47.812807 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1b1855-741b-4cd7-a2c9-20a96f7d2a41 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:47.822550 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1015/2021] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:42:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:42:48.088156 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80666b1e-f3e8-44cc-a8e5-012077a614e3 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:48.089641 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-80666b1e-f3e8-44cc-a8e5-012077a614e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:48.104440 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80666b1e-f3e8-44cc-a8e5-012077a614e3 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:48.109240 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d85e40-c763-4f5c-a281-9030fa9c0af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:48.109907 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d85e40-c763-4f5c-a281-9030fa9c0af3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:48.110244 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d85e40-c763-4f5c-a281-9030fa9c0af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:48.110660 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d85e40-c763-4f5c-a281-9030fa9c0af3 admin admin] PATH_INFO: `/v3/projects/23605b91ac6e48aeac842caa26a5d2be` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:48.111642 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d85e40-c763-4f5c-a281-9030fa9c0af3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23605b91ac6e48aeac842caa26a5d2be)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:48.114037 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d85e40-c763-4f5c-a281-9030fa9c0af3 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:48.162158 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d7d85e40-c763-4f5c-a281-9030fa9c0af3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23605b91ac6e48aeac842caa26a5d2be', 'request_id': 'req-d7d85e40-c763-4f5c-a281-9030fa9c0af3'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:48.168276 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d7d85e40-c763-4f5c-a281-9030fa9c0af3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23605b91ac6e48aeac842caa26a5d2be', 'request_id': 'req-d7d85e40-c763-4f5c-a281-9030fa9c0af3'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:48.177702 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d7d85e40-c763-4f5c-a281-9030fa9c0af3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23605b91ac6e48aeac842caa26a5d2be', 'request_id': 'req-d7d85e40-c763-4f5c-a281-9030fa9c0af3'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:48.194982 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d7d85e40-c763-4f5c-a281-9030fa9c0af3 admin admin] The token cache is being invalidate because project 23605b91ac6e48aeac842caa26a5d2be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:42:48.195272 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d7d85e40-c763-4f5c-a281-9030fa9c0af3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:48.195885 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d7d85e40-c763-4f5c-a281-9030fa9c0af3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:48.197952 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1007/2022] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:42:48 2022] DELETE /identity/v3/projects/23605b91ac6e48aeac842caa26a5d2be => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:42:48.202228 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b1855-741b-4cd7-a2c9-20a96f7d2a41 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:48.211975 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b1855-741b-4cd7-a2c9-20a96f7d2a41 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:48.267539 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e253041-77f9-4a04-aeca-cd78d8caaebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:48.268303 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e253041-77f9-4a04-aeca-cd78d8caaebc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:48.268811 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e253041-77f9-4a04-aeca-cd78d8caaebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:48.269262 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e253041-77f9-4a04-aeca-cd78d8caaebc admin admin] PATH_INFO: `/v3/users/c5374286686147119b9f6edde7ac556d` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:48.270786 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e253041-77f9-4a04-aeca-cd78d8caaebc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5374286686147119b9f6edde7ac556d)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:48.272943 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e253041-77f9-4a04-aeca-cd78d8caaebc admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:48.327156 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5e253041-77f9-4a04-aeca-cd78d8caaebc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5374286686147119b9f6edde7ac556d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:48.334222 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5e253041-77f9-4a04-aeca-cd78d8caaebc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5374286686147119b9f6edde7ac556d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:48.348757 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5e253041-77f9-4a04-aeca-cd78d8caaebc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5374286686147119b9f6edde7ac556d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:48.356816 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5e253041-77f9-4a04-aeca-cd78d8caaebc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5374286686147119b9f6edde7ac556d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:48.357437 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5e253041-77f9-4a04-aeca-cd78d8caaebc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5374286686147119b9f6edde7ac556d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:48.363445 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1016/2023] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:42:48 2022] DELETE /identity/v3/users/c5374286686147119b9f6edde7ac556d => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:42:48.633322 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d85e40-c763-4f5c-a281-9030fa9c0af3 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:48.635444 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d7d85e40-c763-4f5c-a281-9030fa9c0af3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:48.652536 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d85e40-c763-4f5c-a281-9030fa9c0af3 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:48.709291 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca8ac42-e597-416a-9669-4407e8b2c7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:48.709922 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8ac42-e597-416a-9669-4407e8b2c7c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:48.710197 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8ac42-e597-416a-9669-4407e8b2c7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:48.710462 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8ac42-e597-416a-9669-4407e8b2c7c8 admin admin] PATH_INFO: `/v3/projects/f1adf863d3b6482fa6646f408a46ca19` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:48.711569 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca8ac42-e597-416a-9669-4407e8b2c7c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1adf863d3b6482fa6646f408a46ca19)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:48.714001 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca8ac42-e597-416a-9669-4407e8b2c7c8 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:48.758839 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-eca8ac42-e597-416a-9669-4407e8b2c7c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1adf863d3b6482fa6646f408a46ca19', 'request_id': 'req-eca8ac42-e597-416a-9669-4407e8b2c7c8'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:48.765297 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-eca8ac42-e597-416a-9669-4407e8b2c7c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1adf863d3b6482fa6646f408a46ca19', 'request_id': 'req-eca8ac42-e597-416a-9669-4407e8b2c7c8'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:48.770078 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:42:48.771279 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-eca8ac42-e597-416a-9669-4407e8b2c7c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1adf863d3b6482fa6646f408a46ca19', 'request_id': 'req-eca8ac42-e597-416a-9669-4407e8b2c7c8'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:48.785838 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-eca8ac42-e597-416a-9669-4407e8b2c7c8 admin admin] The token cache is being invalidate because project f1adf863d3b6482fa6646f408a46ca19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:42:48.786270 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-eca8ac42-e597-416a-9669-4407e8b2c7c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:48.786904 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-eca8ac42-e597-416a-9669-4407e8b2c7c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:48.788474 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1008/2024] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:42:48 2022] DELETE /identity/v3/projects/f1adf863d3b6482fa6646f408a46ca19 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:42:48.794328 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c2f734-9ac1-44f1-9c85-507aad74f879 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:48.794831 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c2f734-9ac1-44f1-9c85-507aad74f879 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:48.794928 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c2f734-9ac1-44f1-9c85-507aad74f879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:48.815798 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-56c2f734-9ac1-44f1-9c85-507aad74f879 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:42:48.817525 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-56c2f734-9ac1-44f1-9c85-507aad74f879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:48.879481 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1017/2025] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:42:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:42:48.887183 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc4c12-11fe-4f49-b8cd-4e07cb413e56 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:48.887468 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc4c12-11fe-4f49-b8cd-4e07cb413e56 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:48.887741 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc4c12-11fe-4f49-b8cd-4e07cb413e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:48.908537 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-05bc4c12-11fe-4f49-b8cd-4e07cb413e56 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:42:48.909743 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-05bc4c12-11fe-4f49-b8cd-4e07cb413e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:48.918255 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1009/2026] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:42:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:42:48.922412 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c2f734-9ac1-44f1-9c85-507aad74f879 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:48.928641 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:42:48.932799 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c2f734-9ac1-44f1-9c85-507aad74f879 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:48.939471 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adffd20-2aef-4298-b6a5-3d4618a63faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:48.940136 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adffd20-2aef-4298-b6a5-3d4618a63faa admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:48.940586 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adffd20-2aef-4298-b6a5-3d4618a63faa admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:48.940865 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adffd20-2aef-4298-b6a5-3d4618a63faa admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:48.941531 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adffd20-2aef-4298-b6a5-3d4618a63faa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:42:48.941833 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adffd20-2aef-4298-b6a5-3d4618a63faa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:48.943427 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adffd20-2aef-4298-b6a5-3d4618a63faa admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:48.955386 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1018/2027] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:42:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:42:48.959796 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bc4c12-11fe-4f49-b8cd-4e07cb413e56 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:48.969623 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bc4c12-11fe-4f49-b8cd-4e07cb413e56 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:48.974233 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5ade0-bc15-4db8-94c0-edff6725dbaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:48.974817 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5ade0-bc15-4db8-94c0-edff6725dbaa admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:48.975088 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5ade0-bc15-4db8-94c0-edff6725dbaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:48.975350 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5ade0-bc15-4db8-94c0-edff6725dbaa admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:48.975966 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a5ade0-bc15-4db8-94c0-edff6725dbaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:48.978548 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a5ade0-bc15-4db8-94c0-edff6725dbaa admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:49.004177 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1010/2028] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:42:48 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:42:49.008272 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adffd20-2aef-4298-b6a5-3d4618a63faa admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:49.019379 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adffd20-2aef-4298-b6a5-3d4618a63faa admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:49.076585 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd8a461-e60e-4cbe-8419-828c8a296cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:49.077188 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8a461-e60e-4cbe-8419-828c8a296cfe admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:49.077468 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8a461-e60e-4cbe-8419-828c8a296cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:49.077743 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8a461-e60e-4cbe-8419-828c8a296cfe admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:49.078363 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd8a461-e60e-4cbe-8419-828c8a296cfe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:49.080266 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd8a461-e60e-4cbe-8419-828c8a296cfe admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:49.101202 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1019/2029] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:42:49 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:42:49.106879 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5ade0-bc15-4db8-94c0-edff6725dbaa admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:49.117176 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5ade0-bc15-4db8-94c0-edff6725dbaa admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:49.121711 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd718e5-1a51-4ac5-82d1-fe17278ce7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:49.122298 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd718e5-1a51-4ac5-82d1-fe17278ce7c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:49.122571 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd718e5-1a51-4ac5-82d1-fe17278ce7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:49.122836 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd718e5-1a51-4ac5-82d1-fe17278ce7c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:49.123467 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd718e5-1a51-4ac5-82d1-fe17278ce7c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:49.125093 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd718e5-1a51-4ac5-82d1-fe17278ce7c3 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:49.136031 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1011/2030] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:42:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 09:42:49.139845 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd8a461-e60e-4cbe-8419-828c8a296cfe admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:49.150506 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd8a461-e60e-4cbe-8419-828c8a296cfe admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:49.155215 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd35bf-aae0-4c69-8a01-96f8900eeedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:49.155813 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd35bf-aae0-4c69-8a01-96f8900eeedc admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:49.156093 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd35bf-aae0-4c69-8a01-96f8900eeedc admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:49.156376 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd35bf-aae0-4c69-8a01-96f8900eeedc admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:49.157028 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbd35bf-aae0-4c69-8a01-96f8900eeedc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:49.158611 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbd35bf-aae0-4c69-8a01-96f8900eeedc admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:49.169122 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1020/2031] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:42:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:42:49.173257 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd718e5-1a51-4ac5-82d1-fe17278ce7c3 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:49.184417 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd718e5-1a51-4ac5-82d1-fe17278ce7c3 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:49.190018 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410453e0-b2c7-4bf9-9386-e4304016820e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:49.190690 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410453e0-b2c7-4bf9-9386-e4304016820e admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:49.190965 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410453e0-b2c7-4bf9-9386-e4304016820e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:49.191235 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410453e0-b2c7-4bf9-9386-e4304016820e admin admin] PATH_INFO: `/v3/projects/bc63c42396ea4366a0adf74c61a4e444/users/57220932ff5d4d3594f286ff26804567/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:49.208569 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410453e0-b2c7-4bf9-9386-e4304016820e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc63c42396ea4366a0adf74c61a4e444, user_id=57220932ff5d4d3594f286ff26804567, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:49.211445 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410453e0-b2c7-4bf9-9386-e4304016820e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:49.227366 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1012/2032] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:42:49 2022] PUT /identity/v3/projects/bc63c42396ea4366a0adf74c61a4e444/users/57220932ff5d4d3594f286ff26804567/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:42:49.237695 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4c72f-a427-4db9-9422-edd667a41e7f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:49.238049 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4c72f-a427-4db9-9422-edd667a41e7f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:49.238461 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4c72f-a427-4db9-9422-edd667a41e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:49.274652 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-3bb4c72f-a427-4db9-9422-edd667a41e7f None None] MFA Rules not processed for user `57220932ff5d4d3594f286ff26804567`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:42:49.276427 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3bb4c72f-a427-4db9-9422-edd667a41e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:49.352374 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1021/2033] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:42:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:42:49.357092 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410453e0-b2c7-4bf9-9386-e4304016820e admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:49.421827 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410453e0-b2c7-4bf9-9386-e4304016820e admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:49.426455 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2c3f61-012c-402a-bbf3-f83fdb9df36d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:49.427059 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c3f61-012c-402a-bbf3-f83fdb9df36d admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:49.427334 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c3f61-012c-402a-bbf3-f83fdb9df36d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:49.427602 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c3f61-012c-402a-bbf3-f83fdb9df36d admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:49.428220 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2c3f61-012c-402a-bbf3-f83fdb9df36d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:49.430203 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2c3f61-012c-402a-bbf3-f83fdb9df36d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:49.459373 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1013/2034] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:42:49 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:42:49.464345 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb4c72f-a427-4db9-9422-edd667a41e7f None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:49.478337 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb4c72f-a427-4db9-9422-edd667a41e7f None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:49.549890 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bf0bb5-be79-4a51-acb8-10b2d6ea2e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:49.550637 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf0bb5-be79-4a51-acb8-10b2d6ea2e21 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:49.550985 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf0bb5-be79-4a51-acb8-10b2d6ea2e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:49.551314 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf0bb5-be79-4a51-acb8-10b2d6ea2e21 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:49.552036 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bf0bb5-be79-4a51-acb8-10b2d6ea2e21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:49.554278 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bf0bb5-be79-4a51-acb8-10b2d6ea2e21 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:49.568779 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:42:49.586229 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1022/2035] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:42:49 2022] POST /identity/v3/users => generated 300 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:42:49.590719 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2c3f61-012c-402a-bbf3-f83fdb9df36d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:49.603628 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2c3f61-012c-402a-bbf3-f83fdb9df36d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:49.616669 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726dc6c2-ff4d-46fd-be05-3040e73907fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:49.617240 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726dc6c2-ff4d-46fd-be05-3040e73907fc admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:49.617519 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726dc6c2-ff4d-46fd-be05-3040e73907fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:49.617792 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726dc6c2-ff4d-46fd-be05-3040e73907fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:49.618999 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726dc6c2-ff4d-46fd-be05-3040e73907fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:49.620635 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726dc6c2-ff4d-46fd-be05-3040e73907fc admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:49.638193 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1014/2036] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:42:49 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:42:49.642382 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bf0bb5-be79-4a51-acb8-10b2d6ea2e21 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:49.653190 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bf0bb5-be79-4a51-acb8-10b2d6ea2e21 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:49.668644 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86e9335-5f43-4beb-b583-2e7913ceeebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:49.669502 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e9335-5f43-4beb-b583-2e7913ceeebf admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:49.669837 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e9335-5f43-4beb-b583-2e7913ceeebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:49.670113 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e9335-5f43-4beb-b583-2e7913ceeebf admin admin] PATH_INFO: `/v3/projects/fd16797b69664dbbac5947e207804a45/users/ff739b19f4bf4ad89fa947a50d3dbdbc/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:49.690253 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86e9335-5f43-4beb-b583-2e7913ceeebf admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd16797b69664dbbac5947e207804a45, user_id=ff739b19f4bf4ad89fa947a50d3dbdbc, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:49.700592 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86e9335-5f43-4beb-b583-2e7913ceeebf admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:49.719856 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1023/2037] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:42:49 2022] PUT /identity/v3/projects/fd16797b69664dbbac5947e207804a45/users/ff739b19f4bf4ad89fa947a50d3dbdbc/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:42:49.723943 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726dc6c2-ff4d-46fd-be05-3040e73907fc admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:49.735831 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726dc6c2-ff4d-46fd-be05-3040e73907fc admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:49.799904 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1c4427-9f7b-418b-a1b2-2da721e9ba61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:49.800545 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1c4427-9f7b-418b-a1b2-2da721e9ba61 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:49.801376 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1c4427-9f7b-418b-a1b2-2da721e9ba61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:49.801376 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1c4427-9f7b-418b-a1b2-2da721e9ba61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:49.801848 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1c4427-9f7b-418b-a1b2-2da721e9ba61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:49.803489 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1c4427-9f7b-418b-a1b2-2da721e9ba61 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:49.805057 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:42:49.817910 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1015/2038] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:42:49 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 09:42:49.822287 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86e9335-5f43-4beb-b583-2e7913ceeebf admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:49.833041 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86e9335-5f43-4beb-b583-2e7913ceeebf admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:49.837944 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cad102d-e547-47a5-b381-9424979097d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:49.838742 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad102d-e547-47a5-b381-9424979097d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:49.839137 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad102d-e547-47a5-b381-9424979097d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:49.839409 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad102d-e547-47a5-b381-9424979097d9 admin admin] PATH_INFO: `/v3/projects/fd16797b69664dbbac5947e207804a45/users/ff739b19f4bf4ad89fa947a50d3dbdbc/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:49.842128 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cad102d-e547-47a5-b381-9424979097d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd16797b69664dbbac5947e207804a45, user_id=ff739b19f4bf4ad89fa947a50d3dbdbc, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:49.845873 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cad102d-e547-47a5-b381-9424979097d9 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:49.867451 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1024/2039] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:42:49 2022] PUT /identity/v3/projects/fd16797b69664dbbac5947e207804a45/users/ff739b19f4bf4ad89fa947a50d3dbdbc/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:42:49.878428 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42823626-aeb0-4a85-8cdc-539c1c20057e None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:49.878698 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42823626-aeb0-4a85-8cdc-539c1c20057e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:49.878906 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42823626-aeb0-4a85-8cdc-539c1c20057e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:49.908812 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-42823626-aeb0-4a85-8cdc-539c1c20057e None None] MFA Rules not processed for user `ff739b19f4bf4ad89fa947a50d3dbdbc`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:42:49.910504 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-42823626-aeb0-4a85-8cdc-539c1c20057e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:49.932171 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182a422-aa32-4d82-a810-0980319cbe4c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:49.932428 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182a422-aa32-4d82-a810-0980319cbe4c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:49.932700 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182a422-aa32-4d82-a810-0980319cbe4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:49.954148 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-5182a422-aa32-4d82-a810-0980319cbe4c None None] MFA Rules not processed for user `7f389510a6a044d8a7487561b169544b`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:42:49.955388 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5182a422-aa32-4d82-a810-0980319cbe4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:50.012308 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1016/2040] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:42:49 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:42:50.016480 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1025/2041] 10.222.0.70 () {58 vars in 1009 bytes} [Thu Oct 27 09:42:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:42:50.024836 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42823626-aeb0-4a85-8cdc-539c1c20057e None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:50.026359 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-42823626-aeb0-4a85-8cdc-539c1c20057e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:50.028749 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5182a422-aa32-4d82-a810-0980319cbe4c None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:50.030114 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5182a422-aa32-4d82-a810-0980319cbe4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:50.084936 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42823626-aeb0-4a85-8cdc-539c1c20057e None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:50.089255 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e7a7f7-492d-4e3d-b7be-a14dbd024717 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:50.089861 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7a7f7-492d-4e3d-b7be-a14dbd024717 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:50.090138 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7a7f7-492d-4e3d-b7be-a14dbd024717 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:50.090408 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7a7f7-492d-4e3d-b7be-a14dbd024717 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:50.099562 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e7a7f7-492d-4e3d-b7be-a14dbd024717 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:50.101104 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e7a7f7-492d-4e3d-b7be-a14dbd024717 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:50.103799 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5182a422-aa32-4d82-a810-0980319cbe4c None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:50.108881 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d184d1-0290-42dd-9d6b-23092958b681 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eecf79ea4700496c89b1a67c614b34d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:50.109479 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d184d1-0290-42dd-9d6b-23092958b681 service swift] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:50.109752 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d184d1-0290-42dd-9d6b-23092958b681 service swift] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:50.110018 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d184d1-0290-42dd-9d6b-23092958b681 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:50.111327 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1017/2042] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:42:50 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:42:50.118615 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d184d1-0290-42dd-9d6b-23092958b681 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:50.120242 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d184d1-0290-42dd-9d6b-23092958b681 service swift] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:50.129089 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1026/2043] 10.222.0.70 () {60 vars in 1384 bytes} [Thu Oct 27 09:42:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 27 09:42:50.140046 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e7a7f7-492d-4e3d-b7be-a14dbd024717 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:50.234106 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e7a7f7-492d-4e3d-b7be-a14dbd024717 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:50.239007 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c3cb19-2f4a-4e38-8e12-3f9d4d78657c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:50.239705 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3cb19-2f4a-4e38-8e12-3f9d4d78657c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:50.239978 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3cb19-2f4a-4e38-8e12-3f9d4d78657c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:50.240250 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3cb19-2f4a-4e38-8e12-3f9d4d78657c admin admin] PATH_INFO: `/v3/users/57220932ff5d4d3594f286ff26804567` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:50.241435 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c3cb19-2f4a-4e38-8e12-3f9d4d78657c admin admin] RBAC: Authorizing `identity:delete_user(user_id=57220932ff5d4d3594f286ff26804567)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:50.244406 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c3cb19-2f4a-4e38-8e12-3f9d4d78657c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:50.282304 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:42:50.318327 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-45c3cb19-2f4a-4e38-8e12-3f9d4d78657c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57220932ff5d4d3594f286ff26804567'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:50.326308 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-45c3cb19-2f4a-4e38-8e12-3f9d4d78657c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57220932ff5d4d3594f286ff26804567'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:50.364240 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-45c3cb19-2f4a-4e38-8e12-3f9d4d78657c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57220932ff5d4d3594f286ff26804567'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:50.377680 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-45c3cb19-2f4a-4e38-8e12-3f9d4d78657c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57220932ff5d4d3594f286ff26804567'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:50.378366 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-45c3cb19-2f4a-4e38-8e12-3f9d4d78657c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57220932ff5d4d3594f286ff26804567'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:50.388353 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1018/2044] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:42:50 2022] DELETE /identity/v3/users/57220932ff5d4d3594f286ff26804567 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:42:50.392901 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7173a0-28e4-4c50-9899-1743c6995592 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:50.393187 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7173a0-28e4-4c50-9899-1743c6995592 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:50.393460 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7173a0-28e4-4c50-9899-1743c6995592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:50.417549 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-5d7173a0-28e4-4c50-9899-1743c6995592 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:42:50.418876 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5d7173a0-28e4-4c50-9899-1743c6995592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:50.471096 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:42:50.496984 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1027/2045] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:42:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:42:50.507276 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c3cb19-2f4a-4e38-8e12-3f9d4d78657c admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:50.509460 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-45c3cb19-2f4a-4e38-8e12-3f9d4d78657c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:50.609792 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c3cb19-2f4a-4e38-8e12-3f9d4d78657c admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:50.618472 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32eeff1-846b-4328-9597-646f771d4280 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:50.619211 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32eeff1-846b-4328-9597-646f771d4280 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:50.619607 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32eeff1-846b-4328-9597-646f771d4280 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:50.619969 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32eeff1-846b-4328-9597-646f771d4280 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:50.632412 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32eeff1-846b-4328-9597-646f771d4280 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:50.635274 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32eeff1-846b-4328-9597-646f771d4280 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:50.655583 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1019/2046] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:42:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:42:51.121784 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7173a0-28e4-4c50-9899-1743c6995592 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:51.127142 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5d7173a0-28e4-4c50-9899-1743c6995592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:51.147670 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7173a0-28e4-4c50-9899-1743c6995592 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:51.153416 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962bfb06-8f63-48f8-a1b1-cda122be0f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:51.154058 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962bfb06-8f63-48f8-a1b1-cda122be0f4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:51.154331 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962bfb06-8f63-48f8-a1b1-cda122be0f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:51.154603 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962bfb06-8f63-48f8-a1b1-cda122be0f4f admin admin] PATH_INFO: `/v3/projects/bc63c42396ea4366a0adf74c61a4e444` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:51.156878 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962bfb06-8f63-48f8-a1b1-cda122be0f4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc63c42396ea4366a0adf74c61a4e444)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:51.158778 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962bfb06-8f63-48f8-a1b1-cda122be0f4f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:51.221005 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-962bfb06-8f63-48f8-a1b1-cda122be0f4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc63c42396ea4366a0adf74c61a4e444', 'request_id': 'req-962bfb06-8f63-48f8-a1b1-cda122be0f4f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:51.226793 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-962bfb06-8f63-48f8-a1b1-cda122be0f4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc63c42396ea4366a0adf74c61a4e444', 'request_id': 'req-962bfb06-8f63-48f8-a1b1-cda122be0f4f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:51.240179 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-962bfb06-8f63-48f8-a1b1-cda122be0f4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc63c42396ea4366a0adf74c61a4e444', 'request_id': 'req-962bfb06-8f63-48f8-a1b1-cda122be0f4f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:51.254096 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-962bfb06-8f63-48f8-a1b1-cda122be0f4f admin admin] The token cache is being invalidate because project bc63c42396ea4366a0adf74c61a4e444 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:42:51.254611 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-962bfb06-8f63-48f8-a1b1-cda122be0f4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:51.255260 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-962bfb06-8f63-48f8-a1b1-cda122be0f4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:51.260037 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1028/2047] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:42:51 2022] DELETE /identity/v3/projects/bc63c42396ea4366a0adf74c61a4e444 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:42:51.261305 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32eeff1-846b-4328-9597-646f771d4280 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:51.278025 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32eeff1-846b-4328-9597-646f771d4280 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:51.353330 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b11c629-b2d6-436e-9235-db37750045bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:51.353975 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11c629-b2d6-436e-9235-db37750045bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:51.354248 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11c629-b2d6-436e-9235-db37750045bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:51.354655 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11c629-b2d6-436e-9235-db37750045bd admin admin] PATH_INFO: `/v3/users/ff739b19f4bf4ad89fa947a50d3dbdbc` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:51.355782 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b11c629-b2d6-436e-9235-db37750045bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff739b19f4bf4ad89fa947a50d3dbdbc)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:51.357733 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b11c629-b2d6-436e-9235-db37750045bd admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:51.413924 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-1b11c629-b2d6-436e-9235-db37750045bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff739b19f4bf4ad89fa947a50d3dbdbc'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:51.416070 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:42:51.427149 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-1b11c629-b2d6-436e-9235-db37750045bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff739b19f4bf4ad89fa947a50d3dbdbc'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:51.444602 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-1b11c629-b2d6-436e-9235-db37750045bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff739b19f4bf4ad89fa947a50d3dbdbc'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:51.458095 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-1b11c629-b2d6-436e-9235-db37750045bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff739b19f4bf4ad89fa947a50d3dbdbc'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:51.459003 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-1b11c629-b2d6-436e-9235-db37750045bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff739b19f4bf4ad89fa947a50d3dbdbc'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:51.465519 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1020/2048] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:42:51 2022] DELETE /identity/v3/users/ff739b19f4bf4ad89fa947a50d3dbdbc => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:42:51.846940 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962bfb06-8f63-48f8-a1b1-cda122be0f4f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:51.848458 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-962bfb06-8f63-48f8-a1b1-cda122be0f4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:51.891173 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962bfb06-8f63-48f8-a1b1-cda122be0f4f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:51.960558 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476180df-e128-43dc-aae4-3a29733def93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:51.961228 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476180df-e128-43dc-aae4-3a29733def93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:51.961504 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476180df-e128-43dc-aae4-3a29733def93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:51.961772 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476180df-e128-43dc-aae4-3a29733def93 admin admin] PATH_INFO: `/v3/projects/fd16797b69664dbbac5947e207804a45` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:51.963480 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476180df-e128-43dc-aae4-3a29733def93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd16797b69664dbbac5947e207804a45)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:51.965415 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476180df-e128-43dc-aae4-3a29733def93 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:52.033370 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-476180df-e128-43dc-aae4-3a29733def93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd16797b69664dbbac5947e207804a45', 'request_id': 'req-476180df-e128-43dc-aae4-3a29733def93'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:52.040204 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-476180df-e128-43dc-aae4-3a29733def93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd16797b69664dbbac5947e207804a45', 'request_id': 'req-476180df-e128-43dc-aae4-3a29733def93'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:52.053974 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-476180df-e128-43dc-aae4-3a29733def93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd16797b69664dbbac5947e207804a45', 'request_id': 'req-476180df-e128-43dc-aae4-3a29733def93'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:52.065002 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:42:52.070728 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-476180df-e128-43dc-aae4-3a29733def93 admin admin] The token cache is being invalidate because project fd16797b69664dbbac5947e207804a45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:42:52.071079 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-476180df-e128-43dc-aae4-3a29733def93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:52.072683 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-476180df-e128-43dc-aae4-3a29733def93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:52.082244 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1029/2049] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:42:51 2022] DELETE /identity/v3/projects/fd16797b69664dbbac5947e207804a45 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:42:52.086944 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b104af6-e09f-4069-ac9f-d94f934aaf75 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:52.087959 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b104af6-e09f-4069-ac9f-d94f934aaf75 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:52.088236 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b104af6-e09f-4069-ac9f-d94f934aaf75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:52.111972 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-6b104af6-e09f-4069-ac9f-d94f934aaf75 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:42:52.113389 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6b104af6-e09f-4069-ac9f-d94f934aaf75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:52.206454 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1021/2050] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:42:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:42:52.216548 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a6a8c1-1fef-462d-a738-2f5cded33054 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:52.217951 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a6a8c1-1fef-462d-a738-2f5cded33054 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:52.218463 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a6a8c1-1fef-462d-a738-2f5cded33054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:52.244987 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-85a6a8c1-1fef-462d-a738-2f5cded33054 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:42:52.246288 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-85a6a8c1-1fef-462d-a738-2f5cded33054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:52.260586 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1030/2051] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:42:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:42:52.262288 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b104af6-e09f-4069-ac9f-d94f934aaf75 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:52.272555 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b104af6-e09f-4069-ac9f-d94f934aaf75 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:52.278303 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5f53cc-ad7d-4038-a2cb-b9cd48349178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:52.279057 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f53cc-ad7d-4038-a2cb-b9cd48349178 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:52.279611 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f53cc-ad7d-4038-a2cb-b9cd48349178 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:52.279985 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f53cc-ad7d-4038-a2cb-b9cd48349178 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:52.280789 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5f53cc-ad7d-4038-a2cb-b9cd48349178 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:42:52.281190 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5f53cc-ad7d-4038-a2cb-b9cd48349178 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:52.282915 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5f53cc-ad7d-4038-a2cb-b9cd48349178 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:52.300250 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1022/2052] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:42:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:42:52.301930 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a6a8c1-1fef-462d-a738-2f5cded33054 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:52.312399 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a6a8c1-1fef-462d-a738-2f5cded33054 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:52.317251 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cd3913-85d2-4871-b556-429ff5ed3e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:52.317880 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cd3913-85d2-4871-b556-429ff5ed3e48 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:52.318228 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cd3913-85d2-4871-b556-429ff5ed3e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:52.318564 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cd3913-85d2-4871-b556-429ff5ed3e48 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:52.319295 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cd3913-85d2-4871-b556-429ff5ed3e48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:52.321967 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cd3913-85d2-4871-b556-429ff5ed3e48 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:52.356128 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1031/2053] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:42:52 2022] POST /identity/v3/projects => generated 387 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:42:52.359996 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5f53cc-ad7d-4038-a2cb-b9cd48349178 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:52.373946 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5f53cc-ad7d-4038-a2cb-b9cd48349178 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:52.440134 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a32edbc-948b-4b40-86c2-916758896719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:52.440925 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a32edbc-948b-4b40-86c2-916758896719 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:52.441213 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a32edbc-948b-4b40-86c2-916758896719 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:52.441487 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a32edbc-948b-4b40-86c2-916758896719 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:52.442135 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a32edbc-948b-4b40-86c2-916758896719 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:52.444108 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a32edbc-948b-4b40-86c2-916758896719 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:52.472556 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1023/2054] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:42:52 2022] POST /identity/v3/users => generated 294 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:42:52.473975 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cd3913-85d2-4871-b556-429ff5ed3e48 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:52.487750 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cd3913-85d2-4871-b556-429ff5ed3e48 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:52.492926 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b8ece0-921f-4406-9f2b-a3c22c625466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:52.494071 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8ece0-921f-4406-9f2b-a3c22c625466 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:52.494343 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8ece0-921f-4406-9f2b-a3c22c625466 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:52.494607 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8ece0-921f-4406-9f2b-a3c22c625466 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:52.495228 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b8ece0-921f-4406-9f2b-a3c22c625466 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:52.499605 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b8ece0-921f-4406-9f2b-a3c22c625466 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:52.514157 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1032/2055] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:42:52 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:42:52.515566 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a32edbc-948b-4b40-86c2-916758896719 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:52.531017 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a32edbc-948b-4b40-86c2-916758896719 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:52.536922 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0833e6c-b720-48fe-823b-b68894758d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:52.537560 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0833e6c-b720-48fe-823b-b68894758d97 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:52.537831 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0833e6c-b720-48fe-823b-b68894758d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:52.538096 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0833e6c-b720-48fe-823b-b68894758d97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:52.538729 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0833e6c-b720-48fe-823b-b68894758d97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:52.540439 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0833e6c-b720-48fe-823b-b68894758d97 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:52.556338 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1024/2056] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:42:52 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:42:52.558844 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b8ece0-921f-4406-9f2b-a3c22c625466 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:52.570661 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b8ece0-921f-4406-9f2b-a3c22c625466 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:52.580331 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39057384-37bf-4c36-999e-41fd7f750c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:52.581056 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39057384-37bf-4c36-999e-41fd7f750c57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:52.581341 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39057384-37bf-4c36-999e-41fd7f750c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:52.581645 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39057384-37bf-4c36-999e-41fd7f750c57 admin admin] PATH_INFO: `/v3/projects/e2200a7b2d8e4bf4a0863a0e19f78462/users/9057323126d54e9f97243c3ff988d525/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:52.597558 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39057384-37bf-4c36-999e-41fd7f750c57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2200a7b2d8e4bf4a0863a0e19f78462, user_id=9057323126d54e9f97243c3ff988d525, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:52.600250 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39057384-37bf-4c36-999e-41fd7f750c57 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:52.616344 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1033/2057] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:42:52 2022] PUT /identity/v3/projects/e2200a7b2d8e4bf4a0863a0e19f78462/users/9057323126d54e9f97243c3ff988d525/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:42:52.627511 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93968480-d901-4d4d-aba5-b54e5baf9bff None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:52.627795 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93968480-d901-4d4d-aba5-b54e5baf9bff None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:52.628066 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93968480-d901-4d4d-aba5-b54e5baf9bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:52.663510 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-93968480-d901-4d4d-aba5-b54e5baf9bff None None] MFA Rules not processed for user `9057323126d54e9f97243c3ff988d525`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:42:52.664933 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-93968480-d901-4d4d-aba5-b54e5baf9bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:52.685103 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:42:52.748684 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1025/2058] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:42:52 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:42:52.749268 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39057384-37bf-4c36-999e-41fd7f750c57 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:52.831843 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39057384-37bf-4c36-999e-41fd7f750c57 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:52.836608 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb886ec4-5fed-4399-ba9a-f77d862f2802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:52.837358 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb886ec4-5fed-4399-ba9a-f77d862f2802 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:52.837638 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb886ec4-5fed-4399-ba9a-f77d862f2802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:52.837919 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb886ec4-5fed-4399-ba9a-f77d862f2802 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:52.838600 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb886ec4-5fed-4399-ba9a-f77d862f2802 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:52.842501 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb886ec4-5fed-4399-ba9a-f77d862f2802 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:52.878355 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1034/2059] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:42:52 2022] POST /identity/v3/projects => generated 387 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:42:52.882663 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93968480-d901-4d4d-aba5-b54e5baf9bff None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:52.898606 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93968480-d901-4d4d-aba5-b54e5baf9bff None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:52.989375 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac891658-ba1d-43e3-9da7-5a2e15d91f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:52.990009 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac891658-ba1d-43e3-9da7-5a2e15d91f47 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:52.990292 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac891658-ba1d-43e3-9da7-5a2e15d91f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:52.990726 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac891658-ba1d-43e3-9da7-5a2e15d91f47 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:52.991378 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac891658-ba1d-43e3-9da7-5a2e15d91f47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:52.994154 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac891658-ba1d-43e3-9da7-5a2e15d91f47 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:53.018978 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1026/2060] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:42:52 2022] POST /identity/v3/users => generated 300 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:42:53.022672 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb886ec4-5fed-4399-ba9a-f77d862f2802 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:53.033323 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb886ec4-5fed-4399-ba9a-f77d862f2802 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:53.038125 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed58acd3-61ee-49d5-b7f1-d981822aab7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:53.038738 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed58acd3-61ee-49d5-b7f1-d981822aab7d admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:53.039014 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed58acd3-61ee-49d5-b7f1-d981822aab7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:53.039284 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed58acd3-61ee-49d5-b7f1-d981822aab7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:53.039927 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed58acd3-61ee-49d5-b7f1-d981822aab7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:53.042250 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed58acd3-61ee-49d5-b7f1-d981822aab7d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:53.058611 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1035/2061] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:42:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:42:53.062467 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac891658-ba1d-43e3-9da7-5a2e15d91f47 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:53.073195 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac891658-ba1d-43e3-9da7-5a2e15d91f47 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:53.080688 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd5727-9e31-41c6-9ac8-d91553d7297c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:53.082391 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd5727-9e31-41c6-9ac8-d91553d7297c admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:53.082391 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd5727-9e31-41c6-9ac8-d91553d7297c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:53.082391 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd5727-9e31-41c6-9ac8-d91553d7297c admin admin] PATH_INFO: `/v3/projects/1c12a28e9d8e4a8c86295aca60a26536/users/63759e26e860495fabc6206259109858/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:53.105500 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfd5727-9e31-41c6-9ac8-d91553d7297c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c12a28e9d8e4a8c86295aca60a26536, user_id=63759e26e860495fabc6206259109858, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:53.111353 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfd5727-9e31-41c6-9ac8-d91553d7297c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:53.126169 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1027/2062] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:42:53 2022] PUT /identity/v3/projects/1c12a28e9d8e4a8c86295aca60a26536/users/63759e26e860495fabc6206259109858/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:42:53.130169 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed58acd3-61ee-49d5-b7f1-d981822aab7d admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:53.140927 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed58acd3-61ee-49d5-b7f1-d981822aab7d admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:53.208661 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f269cfdd-1b63-4950-9e0a-ada40ff3bdb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:53.209289 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f269cfdd-1b63-4950-9e0a-ada40ff3bdb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:53.209575 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f269cfdd-1b63-4950-9e0a-ada40ff3bdb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:53.209846 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f269cfdd-1b63-4950-9e0a-ada40ff3bdb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:53.210499 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f269cfdd-1b63-4950-9e0a-ada40ff3bdb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:53.212712 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f269cfdd-1b63-4950-9e0a-ada40ff3bdb0 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:53.224534 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1036/2063] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:42:53 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:42:53.229118 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd5727-9e31-41c6-9ac8-d91553d7297c admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:53.240110 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd5727-9e31-41c6-9ac8-d91553d7297c admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:53.244874 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ceed8-a647-4a60-b7e1-e5aa4cb8dd05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:53.245519 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ceed8-a647-4a60-b7e1-e5aa4cb8dd05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:53.245797 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ceed8-a647-4a60-b7e1-e5aa4cb8dd05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:53.246064 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ceed8-a647-4a60-b7e1-e5aa4cb8dd05 admin admin] PATH_INFO: `/v3/projects/1c12a28e9d8e4a8c86295aca60a26536/users/63759e26e860495fabc6206259109858/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:53.248039 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32ceed8-a647-4a60-b7e1-e5aa4cb8dd05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c12a28e9d8e4a8c86295aca60a26536, user_id=63759e26e860495fabc6206259109858, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:53.250715 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32ceed8-a647-4a60-b7e1-e5aa4cb8dd05 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:53.285020 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1028/2064] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:42:53 2022] PUT /identity/v3/projects/1c12a28e9d8e4a8c86295aca60a26536/users/63759e26e860495fabc6206259109858/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:42:53.307840 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea07c0-2025-4f60-a35a-4daf3d2a44de None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:53.308257 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea07c0-2025-4f60-a35a-4daf3d2a44de None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:53.308599 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea07c0-2025-4f60-a35a-4daf3d2a44de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:53.324861 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:42:53.344583 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-62ea07c0-2025-4f60-a35a-4daf3d2a44de None None] MFA Rules not processed for user `63759e26e860495fabc6206259109858`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:42:53.345912 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-62ea07c0-2025-4f60-a35a-4daf3d2a44de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:53.461117 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1037/2065] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:42:53 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:42:53.473315 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ceed8-a647-4a60-b7e1-e5aa4cb8dd05 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:53.476924 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d32ceed8-a647-4a60-b7e1-e5aa4cb8dd05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:53.550626 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ceed8-a647-4a60-b7e1-e5aa4cb8dd05 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:53.555459 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ff862-7863-4b2a-bd68-75e131702ba0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:53.556104 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ff862-7863-4b2a-bd68-75e131702ba0 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:53.556390 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ff862-7863-4b2a-bd68-75e131702ba0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:53.556684 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ff862-7863-4b2a-bd68-75e131702ba0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:53.566701 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4ff862-7863-4b2a-bd68-75e131702ba0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:53.568166 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4ff862-7863-4b2a-bd68-75e131702ba0 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:53.579669 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1029/2066] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:42:53 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:42:53.734418 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ea07c0-2025-4f60-a35a-4daf3d2a44de None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:53.813238 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ea07c0-2025-4f60-a35a-4daf3d2a44de None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:53.817963 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaf863f-d6ab-45a5-af69-de57201bca87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:53.818668 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf863f-d6ab-45a5-af69-de57201bca87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:53.819016 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf863f-d6ab-45a5-af69-de57201bca87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:53.819517 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf863f-d6ab-45a5-af69-de57201bca87 admin admin] PATH_INFO: `/v3/users/9057323126d54e9f97243c3ff988d525` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:53.820595 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaf863f-d6ab-45a5-af69-de57201bca87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9057323126d54e9f97243c3ff988d525)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:53.823780 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaf863f-d6ab-45a5-af69-de57201bca87 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:53.887690 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ffaf863f-d6ab-45a5-af69-de57201bca87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9057323126d54e9f97243c3ff988d525'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:53.897295 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ffaf863f-d6ab-45a5-af69-de57201bca87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9057323126d54e9f97243c3ff988d525'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:53.913185 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ffaf863f-d6ab-45a5-af69-de57201bca87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9057323126d54e9f97243c3ff988d525'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:53.923822 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ffaf863f-d6ab-45a5-af69-de57201bca87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9057323126d54e9f97243c3ff988d525'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:53.924474 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ffaf863f-d6ab-45a5-af69-de57201bca87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9057323126d54e9f97243c3ff988d525'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:53.931409 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1038/2067] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:42:53 2022] DELETE /identity/v3/users/9057323126d54e9f97243c3ff988d525 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:42:53.936610 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c693e-9ebb-4a66-a742-c8fa6478df12 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:53.936610 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c693e-9ebb-4a66-a742-c8fa6478df12 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:53.936610 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c693e-9ebb-4a66-a742-c8fa6478df12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:53.958965 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-680c693e-9ebb-4a66-a742-c8fa6478df12 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:42:53.960123 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-680c693e-9ebb-4a66-a742-c8fa6478df12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:53.971498 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:42:54.066047 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1030/2068] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:42:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:42:54.073036 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaf863f-d6ab-45a5-af69-de57201bca87 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:54.074780 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ffaf863f-d6ab-45a5-af69-de57201bca87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:54.093679 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:42:54.150991 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaf863f-d6ab-45a5-af69-de57201bca87 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:54.155584 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e26504-f3a0-4bbe-a6fd-d8c24d89a18d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:54.156315 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e26504-f3a0-4bbe-a6fd-d8c24d89a18d service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:54.156918 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e26504-f3a0-4bbe-a6fd-d8c24d89a18d service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:54.157488 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e26504-f3a0-4bbe-a6fd-d8c24d89a18d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:54.170707 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e26504-f3a0-4bbe-a6fd-d8c24d89a18d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:54.172249 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e26504-f3a0-4bbe-a6fd-d8c24d89a18d service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:54.189724 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1039/2069] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:42:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:42:54.531873 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c693e-9ebb-4a66-a742-c8fa6478df12 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:54.533476 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-680c693e-9ebb-4a66-a742-c8fa6478df12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:54.557747 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c693e-9ebb-4a66-a742-c8fa6478df12 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:54.563047 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c58e8fa-f726-4aad-b823-14b6a6201ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:54.564010 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58e8fa-f726-4aad-b823-14b6a6201ca8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:54.564329 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58e8fa-f726-4aad-b823-14b6a6201ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:54.564613 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58e8fa-f726-4aad-b823-14b6a6201ca8 admin admin] PATH_INFO: `/v3/projects/e2200a7b2d8e4bf4a0863a0e19f78462` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:54.566173 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c58e8fa-f726-4aad-b823-14b6a6201ca8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2200a7b2d8e4bf4a0863a0e19f78462)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:54.570152 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c58e8fa-f726-4aad-b823-14b6a6201ca8 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:54.647689 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6c58e8fa-f726-4aad-b823-14b6a6201ca8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2200a7b2d8e4bf4a0863a0e19f78462', 'request_id': 'req-6c58e8fa-f726-4aad-b823-14b6a6201ca8'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:54.658002 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6c58e8fa-f726-4aad-b823-14b6a6201ca8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2200a7b2d8e4bf4a0863a0e19f78462', 'request_id': 'req-6c58e8fa-f726-4aad-b823-14b6a6201ca8'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:54.666563 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6c58e8fa-f726-4aad-b823-14b6a6201ca8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2200a7b2d8e4bf4a0863a0e19f78462', 'request_id': 'req-6c58e8fa-f726-4aad-b823-14b6a6201ca8'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:54.698855 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6c58e8fa-f726-4aad-b823-14b6a6201ca8 admin admin] The token cache is being invalidate because project e2200a7b2d8e4bf4a0863a0e19f78462 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:42:54.699206 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6c58e8fa-f726-4aad-b823-14b6a6201ca8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:54.700080 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6c58e8fa-f726-4aad-b823-14b6a6201ca8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:54.701781 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1031/2070] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:42:54 2022] DELETE /identity/v3/projects/e2200a7b2d8e4bf4a0863a0e19f78462 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:42:54.705810 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e26504-f3a0-4bbe-a6fd-d8c24d89a18d service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:54.724806 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e26504-f3a0-4bbe-a6fd-d8c24d89a18d service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:54.784901 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50beed17-32c2-4c66-9ee6-2920d6474af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:54.786979 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50beed17-32c2-4c66-9ee6-2920d6474af1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:54.787269 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50beed17-32c2-4c66-9ee6-2920d6474af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:54.787548 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50beed17-32c2-4c66-9ee6-2920d6474af1 admin admin] PATH_INFO: `/v3/users/63759e26e860495fabc6206259109858` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:54.788758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50beed17-32c2-4c66-9ee6-2920d6474af1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63759e26e860495fabc6206259109858)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:54.791335 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50beed17-32c2-4c66-9ee6-2920d6474af1 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:54.849184 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-50beed17-32c2-4c66-9ee6-2920d6474af1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63759e26e860495fabc6206259109858'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:54.857399 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-50beed17-32c2-4c66-9ee6-2920d6474af1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63759e26e860495fabc6206259109858'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:54.872772 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-50beed17-32c2-4c66-9ee6-2920d6474af1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63759e26e860495fabc6206259109858'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:54.881921 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-50beed17-32c2-4c66-9ee6-2920d6474af1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63759e26e860495fabc6206259109858'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:54.882731 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-50beed17-32c2-4c66-9ee6-2920d6474af1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63759e26e860495fabc6206259109858'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:54.892321 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1040/2071] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:42:54 2022] DELETE /identity/v3/users/63759e26e860495fabc6206259109858 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:42:55.223701 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c58e8fa-f726-4aad-b823-14b6a6201ca8 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:55.225555 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6c58e8fa-f726-4aad-b823-14b6a6201ca8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:55.260507 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c58e8fa-f726-4aad-b823-14b6a6201ca8 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:55.344901 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50530af3-5a1a-4151-b13e-717a80b135ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:55.345654 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50530af3-5a1a-4151-b13e-717a80b135ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:55.345933 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50530af3-5a1a-4151-b13e-717a80b135ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:55.346219 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50530af3-5a1a-4151-b13e-717a80b135ce admin admin] PATH_INFO: `/v3/projects/1c12a28e9d8e4a8c86295aca60a26536` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:55.349213 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50530af3-5a1a-4151-b13e-717a80b135ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c12a28e9d8e4a8c86295aca60a26536)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:55.349376 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50530af3-5a1a-4151-b13e-717a80b135ce admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:55.404880 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:42:55.419006 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-50530af3-5a1a-4151-b13e-717a80b135ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c12a28e9d8e4a8c86295aca60a26536', 'request_id': 'req-50530af3-5a1a-4151-b13e-717a80b135ce'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:55.427169 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-50530af3-5a1a-4151-b13e-717a80b135ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c12a28e9d8e4a8c86295aca60a26536', 'request_id': 'req-50530af3-5a1a-4151-b13e-717a80b135ce'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:55.434460 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-50530af3-5a1a-4151-b13e-717a80b135ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c12a28e9d8e4a8c86295aca60a26536', 'request_id': 'req-50530af3-5a1a-4151-b13e-717a80b135ce'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:55.452264 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-50530af3-5a1a-4151-b13e-717a80b135ce admin admin] The token cache is being invalidate because project 1c12a28e9d8e4a8c86295aca60a26536 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:42:55.452665 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-50530af3-5a1a-4151-b13e-717a80b135ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:55.453358 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-50530af3-5a1a-4151-b13e-717a80b135ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:55.454882 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1032/2072] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:42:55 2022] DELETE /identity/v3/projects/1c12a28e9d8e4a8c86295aca60a26536 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:42:55.459731 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c22422-121b-4aaf-93de-74129aef95b3 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:55.460015 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c22422-121b-4aaf-93de-74129aef95b3 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:55.460286 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c22422-121b-4aaf-93de-74129aef95b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:55.484348 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-03c22422-121b-4aaf-93de-74129aef95b3 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:42:55.485672 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-03c22422-121b-4aaf-93de-74129aef95b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:55.519119 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:42:55.570084 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1041/2073] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:42:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:42:55.582532 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec1332-0226-4b3f-a57f-2be11a1a2bb7 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:55.583085 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec1332-0226-4b3f-a57f-2be11a1a2bb7 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:55.583085 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec1332-0226-4b3f-a57f-2be11a1a2bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:55.619059 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-48ec1332-0226-4b3f-a57f-2be11a1a2bb7 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:42:55.621943 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-48ec1332-0226-4b3f-a57f-2be11a1a2bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:55.636564 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1033/2074] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:42:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:42:55.640234 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c22422-121b-4aaf-93de-74129aef95b3 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:55.654664 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c22422-121b-4aaf-93de-74129aef95b3 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:55.659981 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b6e86a-d912-487f-8110-b40ae65f6e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:55.660830 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6e86a-d912-487f-8110-b40ae65f6e3e admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:55.661299 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6e86a-d912-487f-8110-b40ae65f6e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:55.661656 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6e86a-d912-487f-8110-b40ae65f6e3e admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:55.662616 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b6e86a-d912-487f-8110-b40ae65f6e3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:42:55.663001 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b6e86a-d912-487f-8110-b40ae65f6e3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:55.664954 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b6e86a-d912-487f-8110-b40ae65f6e3e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:55.687062 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1042/2075] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:42:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:42:55.688398 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ec1332-0226-4b3f-a57f-2be11a1a2bb7 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:55.700349 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ec1332-0226-4b3f-a57f-2be11a1a2bb7 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:55.705436 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdfabbe-9248-49b9-95f8-54660e573382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:55.706065 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdfabbe-9248-49b9-95f8-54660e573382 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:55.706978 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdfabbe-9248-49b9-95f8-54660e573382 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:55.707256 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdfabbe-9248-49b9-95f8-54660e573382 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:55.707895 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdfabbe-9248-49b9-95f8-54660e573382 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:55.710665 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdfabbe-9248-49b9-95f8-54660e573382 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:55.758697 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1034/2076] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:42:55 2022] POST /identity/v3/projects => generated 387 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:42:55.760104 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b6e86a-d912-487f-8110-b40ae65f6e3e admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:55.774622 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b6e86a-d912-487f-8110-b40ae65f6e3e admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:55.869779 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e3d10-75f8-4146-bee6-b1c8dc975e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:55.870444 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e3d10-75f8-4146-bee6-b1c8dc975e46 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:55.870719 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e3d10-75f8-4146-bee6-b1c8dc975e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:55.870991 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e3d10-75f8-4146-bee6-b1c8dc975e46 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:55.871638 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82e3d10-75f8-4146-bee6-b1c8dc975e46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:55.873629 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82e3d10-75f8-4146-bee6-b1c8dc975e46 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:55.917522 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1043/2077] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:42:55 2022] POST /identity/v3/users => generated 294 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:42:55.921674 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdfabbe-9248-49b9-95f8-54660e573382 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:55.943685 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdfabbe-9248-49b9-95f8-54660e573382 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:55.949162 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f84eae4-2fe9-4445-b567-c754fc34592a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:55.950143 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84eae4-2fe9-4445-b567-c754fc34592a admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:55.950613 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84eae4-2fe9-4445-b567-c754fc34592a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:55.951020 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84eae4-2fe9-4445-b567-c754fc34592a admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:55.951761 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f84eae4-2fe9-4445-b567-c754fc34592a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:55.954682 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f84eae4-2fe9-4445-b567-c754fc34592a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:55.971995 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1035/2078] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:42:55 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:42:55.973333 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e3d10-75f8-4146-bee6-b1c8dc975e46 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:55.985238 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e3d10-75f8-4146-bee6-b1c8dc975e46 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:55.991349 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c62334-028f-4415-925a-abd15942aef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:55.991973 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c62334-028f-4415-925a-abd15942aef6 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:55.992283 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c62334-028f-4415-925a-abd15942aef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:55.992643 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c62334-028f-4415-925a-abd15942aef6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:55.993833 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c62334-028f-4415-925a-abd15942aef6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:55.995412 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c62334-028f-4415-925a-abd15942aef6 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:56.009013 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1044/2079] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:42:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:42:56.013314 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f84eae4-2fe9-4445-b567-c754fc34592a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:56.027201 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f84eae4-2fe9-4445-b567-c754fc34592a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:56.041210 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705f341a-f2b1-4421-b723-acd5d5a937f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:56.042176 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f341a-f2b1-4421-b723-acd5d5a937f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:56.042716 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f341a-f2b1-4421-b723-acd5d5a937f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:56.043199 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f341a-f2b1-4421-b723-acd5d5a937f3 admin admin] PATH_INFO: `/v3/projects/616e68521bbc496984a846a4a045846c/users/083564e14ed54446b96e9e5723a00faf/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:56.065590 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705f341a-f2b1-4421-b723-acd5d5a937f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=616e68521bbc496984a846a4a045846c, user_id=083564e14ed54446b96e9e5723a00faf, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:56.068564 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705f341a-f2b1-4421-b723-acd5d5a937f3 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:56.084202 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1036/2080] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:42:56 2022] PUT /identity/v3/projects/616e68521bbc496984a846a4a045846c/users/083564e14ed54446b96e9e5723a00faf/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:42:56.095390 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71eb0a-b214-43b8-a23e-a044859390a5 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:56.095673 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71eb0a-b214-43b8-a23e-a044859390a5 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:56.095948 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71eb0a-b214-43b8-a23e-a044859390a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:56.132625 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-8e71eb0a-b214-43b8-a23e-a044859390a5 None None] MFA Rules not processed for user `083564e14ed54446b96e9e5723a00faf`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:42:56.134425 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8e71eb0a-b214-43b8-a23e-a044859390a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:56.226967 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1045/2081] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:42:56 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:42:56.232099 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705f341a-f2b1-4421-b723-acd5d5a937f3 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:56.371098 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705f341a-f2b1-4421-b723-acd5d5a937f3 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:56.377572 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964c3b33-d4b4-42b3-a70a-90dc5040946f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:56.378188 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c3b33-d4b4-42b3-a70a-90dc5040946f admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:56.378478 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c3b33-d4b4-42b3-a70a-90dc5040946f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:56.378741 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c3b33-d4b4-42b3-a70a-90dc5040946f admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:56.379510 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964c3b33-d4b4-42b3-a70a-90dc5040946f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:56.381503 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964c3b33-d4b4-42b3-a70a-90dc5040946f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:56.425732 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1037/2082] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:42:56 2022] POST /identity/v3/projects => generated 385 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:42:56.428665 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71eb0a-b214-43b8-a23e-a044859390a5 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:56.449117 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71eb0a-b214-43b8-a23e-a044859390a5 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:56.504049 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:42:56.565302 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500fb45f-2030-4547-8d66-b102a69d02d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:56.565874 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500fb45f-2030-4547-8d66-b102a69d02d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:56.566211 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500fb45f-2030-4547-8d66-b102a69d02d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:56.566546 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500fb45f-2030-4547-8d66-b102a69d02d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:56.567249 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500fb45f-2030-4547-8d66-b102a69d02d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:56.569733 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500fb45f-2030-4547-8d66-b102a69d02d1 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:56.599107 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1046/2083] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:42:56 2022] POST /identity/v3/users => generated 299 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:42:56.603495 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964c3b33-d4b4-42b3-a70a-90dc5040946f admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:56.614997 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964c3b33-d4b4-42b3-a70a-90dc5040946f admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:56.621043 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e52328-a910-4180-8a54-8c1d4fd27e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:56.621773 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e52328-a910-4180-8a54-8c1d4fd27e2f admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:56.622154 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e52328-a910-4180-8a54-8c1d4fd27e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:56.622524 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e52328-a910-4180-8a54-8c1d4fd27e2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:56.625778 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e52328-a910-4180-8a54-8c1d4fd27e2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:56.627681 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e52328-a910-4180-8a54-8c1d4fd27e2f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:56.644746 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1038/2084] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:42:56 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:42:56.645878 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500fb45f-2030-4547-8d66-b102a69d02d1 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:56.660637 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500fb45f-2030-4547-8d66-b102a69d02d1 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:56.667865 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007e5821-612c-4e84-8af3-a8ab484c1dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:56.668978 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e5821-612c-4e84-8af3-a8ab484c1dae admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:56.669384 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e5821-612c-4e84-8af3-a8ab484c1dae admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:56.669852 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e5821-612c-4e84-8af3-a8ab484c1dae admin admin] PATH_INFO: `/v3/projects/871067cc045e4e5b9299320edf504b6c/users/e688c393ff4b46dba00e8f678191c126/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:56.687568 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007e5821-612c-4e84-8af3-a8ab484c1dae admin admin] RBAC: Authorizing `identity:create_grant(project_id=871067cc045e4e5b9299320edf504b6c, user_id=e688c393ff4b46dba00e8f678191c126, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:56.690770 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007e5821-612c-4e84-8af3-a8ab484c1dae admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:56.709246 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1047/2085] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:42:56 2022] PUT /identity/v3/projects/871067cc045e4e5b9299320edf504b6c/users/e688c393ff4b46dba00e8f678191c126/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:42:56.713629 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e52328-a910-4180-8a54-8c1d4fd27e2f admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:56.727492 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e52328-a910-4180-8a54-8c1d4fd27e2f admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:56.786571 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:42:56.837448 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14904990-1c92-4ff9-b63e-1a2ae668a808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:56.838069 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14904990-1c92-4ff9-b63e-1a2ae668a808 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:56.838345 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14904990-1c92-4ff9-b63e-1a2ae668a808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:56.838617 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14904990-1c92-4ff9-b63e-1a2ae668a808 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:56.839260 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14904990-1c92-4ff9-b63e-1a2ae668a808 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:56.840877 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14904990-1c92-4ff9-b63e-1a2ae668a808 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:56.855262 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1039/2086] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:42:56 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:42:56.859080 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007e5821-612c-4e84-8af3-a8ab484c1dae admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:56.872862 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007e5821-612c-4e84-8af3-a8ab484c1dae admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:56.878015 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5beecc-3502-4c24-b1dc-7d56387d0791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:56.878817 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5beecc-3502-4c24-b1dc-7d56387d0791 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:56.879257 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5beecc-3502-4c24-b1dc-7d56387d0791 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:56.879536 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5beecc-3502-4c24-b1dc-7d56387d0791 admin admin] PATH_INFO: `/v3/projects/871067cc045e4e5b9299320edf504b6c/users/e688c393ff4b46dba00e8f678191c126/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:56.881730 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5beecc-3502-4c24-b1dc-7d56387d0791 admin admin] RBAC: Authorizing `identity:create_grant(project_id=871067cc045e4e5b9299320edf504b6c, user_id=e688c393ff4b46dba00e8f678191c126, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:56.884978 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5beecc-3502-4c24-b1dc-7d56387d0791 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:56.896390 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1048/2087] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:42:56 2022] PUT /identity/v3/projects/871067cc045e4e5b9299320edf504b6c/users/e688c393ff4b46dba00e8f678191c126/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:42:56.907547 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b5a2c-c5ac-4c67-a2c8-e450b274f7cd None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:56.907880 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b5a2c-c5ac-4c67-a2c8-e450b274f7cd None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:56.908202 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b5a2c-c5ac-4c67-a2c8-e450b274f7cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:56.943505 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-3e2b5a2c-c5ac-4c67-a2c8-e450b274f7cd None None] MFA Rules not processed for user `e688c393ff4b46dba00e8f678191c126`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:42:56.944987 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3e2b5a2c-c5ac-4c67-a2c8-e450b274f7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:57.031655 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1040/2088] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:42:56 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:42:57.043197 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5beecc-3502-4c24-b1dc-7d56387d0791 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:57.046052 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8e5beecc-3502-4c24-b1dc-7d56387d0791 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:57.136916 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5beecc-3502-4c24-b1dc-7d56387d0791 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:57.147494 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e713e02a-d368-44f5-a620-ddb43bf1f1a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:57.148346 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713e02a-d368-44f5-a620-ddb43bf1f1a0 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:57.148785 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713e02a-d368-44f5-a620-ddb43bf1f1a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:57.149179 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713e02a-d368-44f5-a620-ddb43bf1f1a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:57.160845 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e713e02a-d368-44f5-a620-ddb43bf1f1a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:57.162599 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e713e02a-d368-44f5-a620-ddb43bf1f1a0 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:57.178495 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1049/2089] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:42:57 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:42:57.990419 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b5a2c-c5ac-4c67-a2c8-e450b274f7cd None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:58.106408 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b5a2c-c5ac-4c67-a2c8-e450b274f7cd None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:58.112034 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ef6f0-857c-4597-95d4-ec63f1ff00ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:58.112812 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ef6f0-857c-4597-95d4-ec63f1ff00ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:58.113162 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ef6f0-857c-4597-95d4-ec63f1ff00ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:58.113523 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ef6f0-857c-4597-95d4-ec63f1ff00ae admin admin] PATH_INFO: `/v3/users/083564e14ed54446b96e9e5723a00faf` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:58.114708 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8ef6f0-857c-4597-95d4-ec63f1ff00ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=083564e14ed54446b96e9e5723a00faf)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:58.116733 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8ef6f0-857c-4597-95d4-ec63f1ff00ae admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:58.180255 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4c8ef6f0-857c-4597-95d4-ec63f1ff00ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '083564e14ed54446b96e9e5723a00faf'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:58.189366 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4c8ef6f0-857c-4597-95d4-ec63f1ff00ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '083564e14ed54446b96e9e5723a00faf'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:58.195930 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:42:58.209477 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4c8ef6f0-857c-4597-95d4-ec63f1ff00ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '083564e14ed54446b96e9e5723a00faf'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:58.218707 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4c8ef6f0-857c-4597-95d4-ec63f1ff00ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '083564e14ed54446b96e9e5723a00faf'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:58.220185 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4c8ef6f0-857c-4597-95d4-ec63f1ff00ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '083564e14ed54446b96e9e5723a00faf'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:58.236468 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1041/2090] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:42:57 2022] DELETE /identity/v3/users/083564e14ed54446b96e9e5723a00faf => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:42:58.238488 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082d478b-119a-4278-9891-0465463577df None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:58.238944 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082d478b-119a-4278-9891-0465463577df None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:58.239041 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082d478b-119a-4278-9891-0465463577df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:58.274246 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-082d478b-119a-4278-9891-0465463577df None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:42:58.275524 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-082d478b-119a-4278-9891-0465463577df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:58.367777 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1050/2091] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:42:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:42:58.377409 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ef6f0-857c-4597-95d4-ec63f1ff00ae admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:58.378986 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4c8ef6f0-857c-4597-95d4-ec63f1ff00ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:58.463676 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ef6f0-857c-4597-95d4-ec63f1ff00ae admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:58.467441 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eb1470-ff5d-4bfc-a317-b46426e6351f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:58.468158 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb1470-ff5d-4bfc-a317-b46426e6351f service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:58.468435 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb1470-ff5d-4bfc-a317-b46426e6351f service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:58.468752 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb1470-ff5d-4bfc-a317-b46426e6351f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:58.479649 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eb1470-ff5d-4bfc-a317-b46426e6351f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:58.481122 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eb1470-ff5d-4bfc-a317-b46426e6351f service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:58.491484 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1042/2092] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:42:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:42:58.814037 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082d478b-119a-4278-9891-0465463577df None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:58.815477 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-082d478b-119a-4278-9891-0465463577df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:58.831980 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082d478b-119a-4278-9891-0465463577df None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:58.845257 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5573ed7a-640b-4107-b1f3-310d0336268f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:58.846178 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5573ed7a-640b-4107-b1f3-310d0336268f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:58.846965 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5573ed7a-640b-4107-b1f3-310d0336268f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:58.846965 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5573ed7a-640b-4107-b1f3-310d0336268f admin admin] PATH_INFO: `/v3/projects/616e68521bbc496984a846a4a045846c` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:58.848118 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5573ed7a-640b-4107-b1f3-310d0336268f admin admin] RBAC: Authorizing `identity:delete_project(project_id=616e68521bbc496984a846a4a045846c)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:58.850343 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5573ed7a-640b-4107-b1f3-310d0336268f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:58.879101 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:42:58.914915 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5573ed7a-640b-4107-b1f3-310d0336268f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '616e68521bbc496984a846a4a045846c', 'request_id': 'req-5573ed7a-640b-4107-b1f3-310d0336268f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:58.922314 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5573ed7a-640b-4107-b1f3-310d0336268f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '616e68521bbc496984a846a4a045846c', 'request_id': 'req-5573ed7a-640b-4107-b1f3-310d0336268f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:58.931089 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5573ed7a-640b-4107-b1f3-310d0336268f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '616e68521bbc496984a846a4a045846c', 'request_id': 'req-5573ed7a-640b-4107-b1f3-310d0336268f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:58.944242 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5573ed7a-640b-4107-b1f3-310d0336268f admin admin] The token cache is being invalidate because project 616e68521bbc496984a846a4a045846c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:42:58.944242 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5573ed7a-640b-4107-b1f3-310d0336268f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:58.945540 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5573ed7a-640b-4107-b1f3-310d0336268f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:58.948372 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1051/2093] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:42:58 2022] DELETE /identity/v3/projects/616e68521bbc496984a846a4a045846c => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:42:58.952853 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eb1470-ff5d-4bfc-a317-b46426e6351f service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:58.964090 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eb1470-ff5d-4bfc-a317-b46426e6351f service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:59.043451 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49483340-357b-422b-9592-bea41f8d46d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:59.044791 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49483340-357b-422b-9592-bea41f8d46d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:59.045773 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49483340-357b-422b-9592-bea41f8d46d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:59.046215 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49483340-357b-422b-9592-bea41f8d46d6 admin admin] PATH_INFO: `/v3/users/e688c393ff4b46dba00e8f678191c126` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:59.047690 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49483340-357b-422b-9592-bea41f8d46d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e688c393ff4b46dba00e8f678191c126)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:59.049920 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49483340-357b-422b-9592-bea41f8d46d6 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:59.143895 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-49483340-357b-422b-9592-bea41f8d46d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e688c393ff4b46dba00e8f678191c126'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:59.161717 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-49483340-357b-422b-9592-bea41f8d46d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e688c393ff4b46dba00e8f678191c126'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:59.178803 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-49483340-357b-422b-9592-bea41f8d46d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e688c393ff4b46dba00e8f678191c126'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:59.207518 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-49483340-357b-422b-9592-bea41f8d46d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e688c393ff4b46dba00e8f678191c126'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:59.209747 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-49483340-357b-422b-9592-bea41f8d46d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e688c393ff4b46dba00e8f678191c126'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:59.226873 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1043/2094] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:42:58 2022] DELETE /identity/v3/users/e688c393ff4b46dba00e8f678191c126 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:42:59.724043 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5573ed7a-640b-4107-b1f3-310d0336268f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:42:59.729864 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5573ed7a-640b-4107-b1f3-310d0336268f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:42:59.766340 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5573ed7a-640b-4107-b1f3-310d0336268f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:42:59.839445 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b78f3-2042-402c-96e8-5a0da6331804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:42:59.840082 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b78f3-2042-402c-96e8-5a0da6331804 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:59.840356 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b78f3-2042-402c-96e8-5a0da6331804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:59.840739 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b78f3-2042-402c-96e8-5a0da6331804 admin admin] PATH_INFO: `/v3/projects/871067cc045e4e5b9299320edf504b6c` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:59.841869 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5b78f3-2042-402c-96e8-5a0da6331804 admin admin] RBAC: Authorizing `identity:delete_project(project_id=871067cc045e4e5b9299320edf504b6c)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:42:59.843771 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5b78f3-2042-402c-96e8-5a0da6331804 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:42:59.899571 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-df5b78f3-2042-402c-96e8-5a0da6331804 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '871067cc045e4e5b9299320edf504b6c', 'request_id': 'req-df5b78f3-2042-402c-96e8-5a0da6331804'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:59.906036 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-df5b78f3-2042-402c-96e8-5a0da6331804 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '871067cc045e4e5b9299320edf504b6c', 'request_id': 'req-df5b78f3-2042-402c-96e8-5a0da6331804'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:59.916618 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-df5b78f3-2042-402c-96e8-5a0da6331804 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '871067cc045e4e5b9299320edf504b6c', 'request_id': 'req-df5b78f3-2042-402c-96e8-5a0da6331804'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:59.929430 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-df5b78f3-2042-402c-96e8-5a0da6331804 admin admin] The token cache is being invalidate because project 871067cc045e4e5b9299320edf504b6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:42:59.929736 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-df5b78f3-2042-402c-96e8-5a0da6331804 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:59.930755 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-df5b78f3-2042-402c-96e8-5a0da6331804 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:42:59.935503 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1052/2095] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:42:59 2022] DELETE /identity/v3/projects/871067cc045e4e5b9299320edf504b6c => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:42:59.938728 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e75f07-1b99-4fb0-bdcc-f2031e906f5b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:42:59.939752 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e75f07-1b99-4fb0-bdcc-f2031e906f5b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:42:59.940381 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e75f07-1b99-4fb0-bdcc-f2031e906f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:42:59.952333 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:42:59.970573 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-e7e75f07-1b99-4fb0-bdcc-f2031e906f5b None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:42:59.972043 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e7e75f07-1b99-4fb0-bdcc-f2031e906f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:00.052021 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a025e-c626-4aed-9ac4-b0f7820fe9be None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:00.052021 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a025e-c626-4aed-9ac4-b0f7820fe9be None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:00.052021 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a025e-c626-4aed-9ac4-b0f7820fe9be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:00.056784 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1044/2096] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:42:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:43:00.075865 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-0e2a025e-c626-4aed-9ac4-b0f7820fe9be None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:43:00.077445 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0e2a025e-c626-4aed-9ac4-b0f7820fe9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:00.091593 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1053/2097] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:43:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:43:00.096754 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e75f07-1b99-4fb0-bdcc-f2031e906f5b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:00.109673 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e75f07-1b99-4fb0-bdcc-f2031e906f5b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:00.116471 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c2aa7-f74b-43ff-83be-39ac7dcebb83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:00.117219 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c2aa7-f74b-43ff-83be-39ac7dcebb83 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:00.117558 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c2aa7-f74b-43ff-83be-39ac7dcebb83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:00.118043 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c2aa7-f74b-43ff-83be-39ac7dcebb83 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:00.118928 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261c2aa7-f74b-43ff-83be-39ac7dcebb83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:43:00.119299 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261c2aa7-f74b-43ff-83be-39ac7dcebb83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:00.121378 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261c2aa7-f74b-43ff-83be-39ac7dcebb83 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:00.135484 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1045/2098] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:43:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:43:00.139593 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2a025e-c626-4aed-9ac4-b0f7820fe9be None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:00.153700 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2a025e-c626-4aed-9ac4-b0f7820fe9be None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:00.158709 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0450e0df-5e5e-4975-9c06-d60cba4d3015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:00.159578 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450e0df-5e5e-4975-9c06-d60cba4d3015 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:00.159927 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450e0df-5e5e-4975-9c06-d60cba4d3015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:00.160267 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450e0df-5e5e-4975-9c06-d60cba4d3015 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:00.161023 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0450e0df-5e5e-4975-9c06-d60cba4d3015 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:00.163296 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0450e0df-5e5e-4975-9c06-d60cba4d3015 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:00.202888 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1054/2099] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:43:00 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:43:00.205689 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c2aa7-f74b-43ff-83be-39ac7dcebb83 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:00.225721 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c2aa7-f74b-43ff-83be-39ac7dcebb83 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:00.335803 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96389c6-1540-40fc-9555-de720c457519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:00.336640 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96389c6-1540-40fc-9555-de720c457519 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:00.337440 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96389c6-1540-40fc-9555-de720c457519 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:00.337715 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96389c6-1540-40fc-9555-de720c457519 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:00.338356 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96389c6-1540-40fc-9555-de720c457519 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:00.340288 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96389c6-1540-40fc-9555-de720c457519 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:00.370038 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0450e0df-5e5e-4975-9c06-d60cba4d3015 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:00.371246 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1046/2100] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:43:00 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:43:00.385310 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0450e0df-5e5e-4975-9c06-d60cba4d3015 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:00.393222 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b04b7-7001-47ef-aa44-86d2c986bd75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:00.394119 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b04b7-7001-47ef-aa44-86d2c986bd75 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:00.394555 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b04b7-7001-47ef-aa44-86d2c986bd75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:00.395165 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b04b7-7001-47ef-aa44-86d2c986bd75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:00.395824 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221b04b7-7001-47ef-aa44-86d2c986bd75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:00.397483 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221b04b7-7001-47ef-aa44-86d2c986bd75 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:00.400808 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:43:00.417878 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96389c6-1540-40fc-9555-de720c457519 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:00.418651 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1055/2101] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:43:00.429062 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96389c6-1540-40fc-9555-de720c457519 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:00.434089 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072682a9-16d3-4dfc-9ddb-6a24026aec31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:00.434706 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072682a9-16d3-4dfc-9ddb-6a24026aec31 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:00.434986 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072682a9-16d3-4dfc-9ddb-6a24026aec31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:00.435258 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072682a9-16d3-4dfc-9ddb-6a24026aec31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:00.435899 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072682a9-16d3-4dfc-9ddb-6a24026aec31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:00.437629 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072682a9-16d3-4dfc-9ddb-6a24026aec31 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:00.458602 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b04b7-7001-47ef-aa44-86d2c986bd75 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:00.468558 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1047/2102] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 09:43:00.473615 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b04b7-7001-47ef-aa44-86d2c986bd75 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:00.480163 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca53511-144d-4742-92e2-7bee01e09eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:00.481090 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca53511-144d-4742-92e2-7bee01e09eb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:00.481429 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca53511-144d-4742-92e2-7bee01e09eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:00.481892 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca53511-144d-4742-92e2-7bee01e09eb2 admin admin] PATH_INFO: `/v3/projects/eaf050de713140879ff4d2e403abe41d/users/98ca55ff7911401e883251bee622887b/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:00.517770 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca53511-144d-4742-92e2-7bee01e09eb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaf050de713140879ff4d2e403abe41d, user_id=98ca55ff7911401e883251bee622887b, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:00.520951 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca53511-144d-4742-92e2-7bee01e09eb2 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:00.553413 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1056/2103] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:43:00 2022] PUT /identity/v3/projects/eaf050de713140879ff4d2e403abe41d/users/98ca55ff7911401e883251bee622887b/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:43:00.555649 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3fbe89-8341-4646-9816-cb5be7b396e3 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:00.555979 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3fbe89-8341-4646-9816-cb5be7b396e3 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:00.556258 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3fbe89-8341-4646-9816-cb5be7b396e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:00.613633 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-3f3fbe89-8341-4646-9816-cb5be7b396e3 None None] MFA Rules not processed for user `98ca55ff7911401e883251bee622887b`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:43:00.614988 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3f3fbe89-8341-4646-9816-cb5be7b396e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:00.709198 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:43:00.736736 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca53511-144d-4742-92e2-7bee01e09eb2 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:00.737289 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1048/2104] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:43:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:43:00.739124 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-fca53511-144d-4742-92e2-7bee01e09eb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:00.812965 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca53511-144d-4742-92e2-7bee01e09eb2 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:00.817455 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c543580-79ae-4410-82c3-f783a4ba8d96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:00.819404 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c543580-79ae-4410-82c3-f783a4ba8d96 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:00.819404 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c543580-79ae-4410-82c3-f783a4ba8d96 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:00.819404 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c543580-79ae-4410-82c3-f783a4ba8d96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:00.829090 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c543580-79ae-4410-82c3-f783a4ba8d96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:00.830841 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c543580-79ae-4410-82c3-f783a4ba8d96 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:00.841288 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1057/2105] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:43:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:43:04.571538 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d6a725-471c-4a56-a88d-1cec56146ba5 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:04.572441 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d6a725-471c-4a56-a88d-1cec56146ba5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:04.573266 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d6a725-471c-4a56-a88d-1cec56146ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:04.602345 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-49d6a725-471c-4a56-a88d-1cec56146ba5 None None] MFA Rules not processed for user `f7d2c6275669404db608d714baeb0065`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:43:04.603563 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-49d6a725-471c-4a56-a88d-1cec56146ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:04.666867 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1049/2106] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:43:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:43:04.969844 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c543580-79ae-4410-82c3-f783a4ba8d96 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:04.971232 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4c543580-79ae-4410-82c3-f783a4ba8d96 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:05.026701 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c543580-79ae-4410-82c3-f783a4ba8d96 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:05.030220 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cc558e-ff60-4768-aed8-5edb957fed1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:05.030817 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc558e-ff60-4768-aed8-5edb957fed1c service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:05.031112 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc558e-ff60-4768-aed8-5edb957fed1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:05.031361 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc558e-ff60-4768-aed8-5edb957fed1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:05.033056 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-44cc558e-ff60-4768-aed8-5edb957fed1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:05.042183 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cc558e-ff60-4768-aed8-5edb957fed1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:05.043588 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cc558e-ff60-4768-aed8-5edb957fed1c service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:05.051737 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1058/2107] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:43:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:43:06.437550 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d6a725-471c-4a56-a88d-1cec56146ba5 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:06.439171 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-49d6a725-471c-4a56-a88d-1cec56146ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:06.536582 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d6a725-471c-4a56-a88d-1cec56146ba5 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:06.542171 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eceda73-4aca-4f64-b7ce-9f609c686351 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe4a6e4b6184826b30ac4fc13ad9b1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:06.542776 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eceda73-4aca-4f64-b7ce-9f609c686351 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:06.543049 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eceda73-4aca-4f64-b7ce-9f609c686351 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:06.543310 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eceda73-4aca-4f64-b7ce-9f609c686351 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:06.545058 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6eceda73-4aca-4f64-b7ce-9f609c686351 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:06.555752 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eceda73-4aca-4f64-b7ce-9f609c686351 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:06.557327 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eceda73-4aca-4f64-b7ce-9f609c686351 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:06.566146 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1050/2108] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:43:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:43:08.577669 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cc558e-ff60-4768-aed8-5edb957fed1c service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:08.719628 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cc558e-ff60-4768-aed8-5edb957fed1c service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:08.725343 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a895a-be4a-4365-b285-3d1b5244594e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:08.726405 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a895a-be4a-4365-b285-3d1b5244594e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:08.727380 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a895a-be4a-4365-b285-3d1b5244594e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:08.727951 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a895a-be4a-4365-b285-3d1b5244594e admin admin] PATH_INFO: `/v3/users/98ca55ff7911401e883251bee622887b` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:08.729601 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058a895a-be4a-4365-b285-3d1b5244594e admin admin] RBAC: Authorizing `identity:delete_user(user_id=98ca55ff7911401e883251bee622887b)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:08.731832 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058a895a-be4a-4365-b285-3d1b5244594e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:08.766320 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:43:08.811419 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-058a895a-be4a-4365-b285-3d1b5244594e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98ca55ff7911401e883251bee622887b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:08.819287 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-058a895a-be4a-4365-b285-3d1b5244594e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98ca55ff7911401e883251bee622887b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:08.862488 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-058a895a-be4a-4365-b285-3d1b5244594e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98ca55ff7911401e883251bee622887b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:08.877117 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-058a895a-be4a-4365-b285-3d1b5244594e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98ca55ff7911401e883251bee622887b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:08.879071 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-058a895a-be4a-4365-b285-3d1b5244594e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98ca55ff7911401e883251bee622887b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:08.889737 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1059/2109] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:43:08 2022] DELETE /identity/v3/users/98ca55ff7911401e883251bee622887b => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:43:08.894400 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd90cd-a66c-4d99-8497-ddb15aca7074 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:08.895290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd90cd-a66c-4d99-8497-ddb15aca7074 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:08.895775 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd90cd-a66c-4d99-8497-ddb15aca7074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:08.918149 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-7cfd90cd-a66c-4d99-8497-ddb15aca7074 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:43:08.919637 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7cfd90cd-a66c-4d99-8497-ddb15aca7074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:09.012464 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1051/2110] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:43:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:43:09.025180 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a895a-be4a-4365-b285-3d1b5244594e admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:09.027145 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-058a895a-be4a-4365-b285-3d1b5244594e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:09.124213 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a895a-be4a-4365-b285-3d1b5244594e admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:09.129349 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a8dc19-97c8-4874-9a86-991f84954403 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:09.130080 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a8dc19-97c8-4874-9a86-991f84954403 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:09.130493 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a8dc19-97c8-4874-9a86-991f84954403 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:09.130861 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a8dc19-97c8-4874-9a86-991f84954403 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:09.141947 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a8dc19-97c8-4874-9a86-991f84954403 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:09.143703 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a8dc19-97c8-4874-9a86-991f84954403 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:09.156751 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1060/2111] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:43:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:43:09.645584 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfd90cd-a66c-4d99-8497-ddb15aca7074 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:09.646996 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7cfd90cd-a66c-4d99-8497-ddb15aca7074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:09.661946 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfd90cd-a66c-4d99-8497-ddb15aca7074 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:09.669045 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab224d43-014f-4608-a49f-f4e0901cd0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:09.670120 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab224d43-014f-4608-a49f-f4e0901cd0ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:09.670635 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab224d43-014f-4608-a49f-f4e0901cd0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:09.671121 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab224d43-014f-4608-a49f-f4e0901cd0ff admin admin] PATH_INFO: `/v3/projects/eaf050de713140879ff4d2e403abe41d` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:09.674711 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab224d43-014f-4608-a49f-f4e0901cd0ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaf050de713140879ff4d2e403abe41d)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:09.675693 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab224d43-014f-4608-a49f-f4e0901cd0ff admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:09.733462 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:43:09.787275 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ab224d43-014f-4608-a49f-f4e0901cd0ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaf050de713140879ff4d2e403abe41d', 'request_id': 'req-ab224d43-014f-4608-a49f-f4e0901cd0ff'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:09.795025 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ab224d43-014f-4608-a49f-f4e0901cd0ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaf050de713140879ff4d2e403abe41d', 'request_id': 'req-ab224d43-014f-4608-a49f-f4e0901cd0ff'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:09.804076 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ab224d43-014f-4608-a49f-f4e0901cd0ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaf050de713140879ff4d2e403abe41d', 'request_id': 'req-ab224d43-014f-4608-a49f-f4e0901cd0ff'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:09.818523 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ab224d43-014f-4608-a49f-f4e0901cd0ff admin admin] The token cache is being invalidate because project eaf050de713140879ff4d2e403abe41d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:43:09.818814 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ab224d43-014f-4608-a49f-f4e0901cd0ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:09.819561 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ab224d43-014f-4608-a49f-f4e0901cd0ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:09.821119 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1052/2112] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:43:09 2022] DELETE /identity/v3/projects/eaf050de713140879ff4d2e403abe41d => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:43:09.826218 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b98e1-7e6b-4c24-bbd3-e3b249d991c6 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:09.826689 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b98e1-7e6b-4c24-bbd3-e3b249d991c6 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:09.826893 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b98e1-7e6b-4c24-bbd3-e3b249d991c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:09.847992 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-e69b98e1-7e6b-4c24-bbd3-e3b249d991c6 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:43:09.849613 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e69b98e1-7e6b-4c24-bbd3-e3b249d991c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:09.917329 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1061/2113] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:43:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:43:09.983036 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d39ef84-e440-424a-a578-1841d62b6499 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:09.983670 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d39ef84-e440-424a-a578-1841d62b6499 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:09.983670 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d39ef84-e440-424a-a578-1841d62b6499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:10.003461 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-3d39ef84-e440-424a-a578-1841d62b6499 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:43:10.004637 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3d39ef84-e440-424a-a578-1841d62b6499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:10.013317 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1053/2114] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:43:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:43:10.017514 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69b98e1-7e6b-4c24-bbd3-e3b249d991c6 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:10.027901 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69b98e1-7e6b-4c24-bbd3-e3b249d991c6 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:10.032447 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda864ad-2a5c-4c51-b3c1-90ace8e34702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:10.033079 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda864ad-2a5c-4c51-b3c1-90ace8e34702 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:10.033366 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda864ad-2a5c-4c51-b3c1-90ace8e34702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:10.033637 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda864ad-2a5c-4c51-b3c1-90ace8e34702 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:10.034273 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda864ad-2a5c-4c51-b3c1-90ace8e34702 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:43:10.034598 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda864ad-2a5c-4c51-b3c1-90ace8e34702 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:10.036185 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda864ad-2a5c-4c51-b3c1-90ace8e34702 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:10.051169 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1062/2115] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:43:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:43:10.052214 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d39ef84-e440-424a-a578-1841d62b6499 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:10.062070 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d39ef84-e440-424a-a578-1841d62b6499 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:10.068093 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3462ce-679e-470d-957b-a43ef7d8b385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:10.068747 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3462ce-679e-470d-957b-a43ef7d8b385 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:10.069022 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3462ce-679e-470d-957b-a43ef7d8b385 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:10.069296 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3462ce-679e-470d-957b-a43ef7d8b385 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:10.069942 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3462ce-679e-470d-957b-a43ef7d8b385 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:10.072414 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3462ce-679e-470d-957b-a43ef7d8b385 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:10.102538 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1054/2116] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:43:10 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:43:10.106781 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda864ad-2a5c-4c51-b3c1-90ace8e34702 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:10.117427 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda864ad-2a5c-4c51-b3c1-90ace8e34702 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:10.155267 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:43:10.200771 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f120cc-8314-4e74-adc9-6bca02fa01cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:10.201678 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f120cc-8314-4e74-adc9-6bca02fa01cb admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:10.202328 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f120cc-8314-4e74-adc9-6bca02fa01cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:10.202683 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f120cc-8314-4e74-adc9-6bca02fa01cb admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:10.203410 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f120cc-8314-4e74-adc9-6bca02fa01cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:10.205586 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f120cc-8314-4e74-adc9-6bca02fa01cb admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:10.236653 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1063/2117] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:43:10 2022] POST /identity/v3/users => generated 286 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:43:10.250026 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3462ce-679e-470d-957b-a43ef7d8b385 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:10.267077 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3462ce-679e-470d-957b-a43ef7d8b385 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:10.279240 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb0169d-1da2-4ca6-ab82-73158e10a870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:10.279240 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb0169d-1da2-4ca6-ab82-73158e10a870 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:10.279240 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb0169d-1da2-4ca6-ab82-73158e10a870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:10.279727 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb0169d-1da2-4ca6-ab82-73158e10a870 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:10.280473 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb0169d-1da2-4ca6-ab82-73158e10a870 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:10.283240 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb0169d-1da2-4ca6-ab82-73158e10a870 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:10.297860 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1055/2118] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:43:10 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:43:10.299095 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f120cc-8314-4e74-adc9-6bca02fa01cb admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:10.309743 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f120cc-8314-4e74-adc9-6bca02fa01cb admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:10.315921 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c709c2c7-687c-4889-bc4a-710646e71d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:10.316698 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c709c2c7-687c-4889-bc4a-710646e71d12 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:10.317100 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c709c2c7-687c-4889-bc4a-710646e71d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:10.317470 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c709c2c7-687c-4889-bc4a-710646e71d12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:10.318214 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c709c2c7-687c-4889-bc4a-710646e71d12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:10.320125 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c709c2c7-687c-4889-bc4a-710646e71d12 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:10.335074 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1064/2119] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:43:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:43:10.336973 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb0169d-1da2-4ca6-ab82-73158e10a870 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:10.347895 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb0169d-1da2-4ca6-ab82-73158e10a870 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:10.353791 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b716a149-cb0f-4d01-bd2d-07dfe9dcbefd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:10.354697 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716a149-cb0f-4d01-bd2d-07dfe9dcbefd admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:10.355062 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716a149-cb0f-4d01-bd2d-07dfe9dcbefd admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:10.355433 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716a149-cb0f-4d01-bd2d-07dfe9dcbefd admin admin] PATH_INFO: `/v3/projects/80470dda88894ca282a2fe303df96ef3/users/70b618ffc3044d919391a60ca2757a3c/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:10.372213 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b716a149-cb0f-4d01-bd2d-07dfe9dcbefd admin admin] RBAC: Authorizing `identity:create_grant(project_id=80470dda88894ca282a2fe303df96ef3, user_id=70b618ffc3044d919391a60ca2757a3c, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:10.375018 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b716a149-cb0f-4d01-bd2d-07dfe9dcbefd admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:10.390960 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1056/2120] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:43:10 2022] PUT /identity/v3/projects/80470dda88894ca282a2fe303df96ef3/users/70b618ffc3044d919391a60ca2757a3c/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:43:10.401514 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c01ccd-c953-40fb-8d94-17b57bdbf10a None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:10.402192 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c01ccd-c953-40fb-8d94-17b57bdbf10a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:10.402496 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c01ccd-c953-40fb-8d94-17b57bdbf10a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:10.437561 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-e2c01ccd-c953-40fb-8d94-17b57bdbf10a None None] MFA Rules not processed for user `70b618ffc3044d919391a60ca2757a3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:43:10.438783 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e2c01ccd-c953-40fb-8d94-17b57bdbf10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:10.521311 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1065/2121] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:43:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:43:10.544725 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b716a149-cb0f-4d01-bd2d-07dfe9dcbefd admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:10.544725 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b716a149-cb0f-4d01-bd2d-07dfe9dcbefd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:10.602201 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b716a149-cb0f-4d01-bd2d-07dfe9dcbefd admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:10.606063 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cb890c-895d-4023-b7c5-336e122cefd5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72b0538f0e42446887b1d95b2fb41fb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:10.606768 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb890c-895d-4023-b7c5-336e122cefd5 service glance] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:10.607120 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb890c-895d-4023-b7c5-336e122cefd5 service glance] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:10.607470 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb890c-895d-4023-b7c5-336e122cefd5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:10.617119 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cb890c-895d-4023-b7c5-336e122cefd5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:10.619240 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cb890c-895d-4023-b7c5-336e122cefd5 service glance] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:10.630821 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1057/2122] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:43:10 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:43:13.991880 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8068a7-4ca6-43f9-9546-eb49089d27fa None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:13.992562 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8068a7-4ca6-43f9-9546-eb49089d27fa None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:13.992562 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8068a7-4ca6-43f9-9546-eb49089d27fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:14.017901 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-3e8068a7-4ca6-43f9-9546-eb49089d27fa None None] MFA Rules not processed for user `7f389510a6a044d8a7487561b169544b`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:43:14.019217 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3e8068a7-4ca6-43f9-9546-eb49089d27fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:14.099773 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1066/2123] 10.222.0.70 () {58 vars in 1009 bytes} [Thu Oct 27 09:43:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:43:14.112821 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cb890c-895d-4023-b7c5-336e122cefd5 service glance] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:14.115144 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-95cb890c-895d-4023-b7c5-336e122cefd5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:14.181427 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:43:14.208148 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cb890c-895d-4023-b7c5-336e122cefd5 service glance] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:14.214831 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8c52d0-d629-4384-817f-c043a1c76b84 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eecf79ea4700496c89b1a67c614b34d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:14.215544 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c52d0-d629-4384-817f-c043a1c76b84 service swift] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:14.215825 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c52d0-d629-4384-817f-c043a1c76b84 service swift] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:14.216104 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c52d0-d629-4384-817f-c043a1c76b84 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:14.232032 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8c52d0-d629-4384-817f-c043a1c76b84 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:14.234729 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8c52d0-d629-4384-817f-c043a1c76b84 service swift] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:14.252632 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1058/2124] 10.222.0.70 () {60 vars in 1384 bytes} [Thu Oct 27 09:43:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 27 09:43:14.327801 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8068a7-4ca6-43f9-9546-eb49089d27fa None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:14.329341 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3e8068a7-4ca6-43f9-9546-eb49089d27fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:14.453460 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:43:14.525842 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8068a7-4ca6-43f9-9546-eb49089d27fa None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:14.539273 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d4257a-f6ad-44c7-ae1b-4b67a449fdde service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe4a6e4b6184826b30ac4fc13ad9b1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:14.540051 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d4257a-f6ad-44c7-ae1b-4b67a449fdde service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:14.540334 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d4257a-f6ad-44c7-ae1b-4b67a449fdde service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:14.540631 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d4257a-f6ad-44c7-ae1b-4b67a449fdde service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:14.546592 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f2d4257a-f6ad-44c7-ae1b-4b67a449fdde service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:14.589614 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d4257a-f6ad-44c7-ae1b-4b67a449fdde service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:14.596727 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d4257a-f6ad-44c7-ae1b-4b67a449fdde service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:14.648260 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1067/2125] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:43:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:43:24.241123 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=9 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:43:24.621744 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=15 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:43:29.344454 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8c52d0-d629-4384-817f-c043a1c76b84 service swift] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:29.345913 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-bd8c52d0-d629-4384-817f-c043a1c76b84 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:29.422157 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8c52d0-d629-4384-817f-c043a1c76b84 service swift] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:29.429480 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be19a50-e57a-4483-b361-ff4e9ec975ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:29.430398 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be19a50-e57a-4483-b361-ff4e9ec975ad service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:29.430911 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be19a50-e57a-4483-b361-ff4e9ec975ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:29.431758 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be19a50-e57a-4483-b361-ff4e9ec975ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:29.436043 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5be19a50-e57a-4483-b361-ff4e9ec975ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:29.521254 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be19a50-e57a-4483-b361-ff4e9ec975ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:29.522730 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be19a50-e57a-4483-b361-ff4e9ec975ad service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:29.536121 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1059/2126] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:43:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:43:33.030057 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d4257a-f6ad-44c7-ae1b-4b67a449fdde service nova] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:33.033867 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f2d4257a-f6ad-44c7-ae1b-4b67a449fdde service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:33.047965 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d4257a-f6ad-44c7-ae1b-4b67a449fdde service nova] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:33.052814 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2074b6fb-fb81-44d2-940f-10741581fdf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:33.053545 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2074b6fb-fb81-44d2-940f-10741581fdf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:33.053915 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2074b6fb-fb81-44d2-940f-10741581fdf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:33.054385 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2074b6fb-fb81-44d2-940f-10741581fdf4 admin admin] PATH_INFO: `/v3/users/286df4c587d7408d988c4739f4ff923a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:33.063441 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2074b6fb-fb81-44d2-940f-10741581fdf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=286df4c587d7408d988c4739f4ff923a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:33.063441 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2074b6fb-fb81-44d2-940f-10741581fdf4 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:33.108367 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2074b6fb-fb81-44d2-940f-10741581fdf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '286df4c587d7408d988c4739f4ff923a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:33.116038 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2074b6fb-fb81-44d2-940f-10741581fdf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '286df4c587d7408d988c4739f4ff923a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:33.129385 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2074b6fb-fb81-44d2-940f-10741581fdf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '286df4c587d7408d988c4739f4ff923a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:33.137282 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2074b6fb-fb81-44d2-940f-10741581fdf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '286df4c587d7408d988c4739f4ff923a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:33.137972 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2074b6fb-fb81-44d2-940f-10741581fdf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '286df4c587d7408d988c4739f4ff923a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:33.144402 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1068/2127] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:43:33 2022] DELETE /identity/v3/users/286df4c587d7408d988c4739f4ff923a => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:43:33.573392 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be19a50-e57a-4483-b361-ff4e9ec975ad service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:33.584005 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be19a50-e57a-4483-b361-ff4e9ec975ad service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:33.642280 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d0e534-7ff0-49f8-946e-9df46dca5ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:33.642909 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0e534-7ff0-49f8-946e-9df46dca5ddd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:33.643194 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0e534-7ff0-49f8-946e-9df46dca5ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:33.643463 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0e534-7ff0-49f8-946e-9df46dca5ddd admin admin] PATH_INFO: `/v3/projects/71d1c4930a1b487eb3dca8fde6ce424a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:33.644634 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d0e534-7ff0-49f8-946e-9df46dca5ddd admin admin] RBAC: Authorizing `identity:delete_project(project_id=71d1c4930a1b487eb3dca8fde6ce424a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:33.646544 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d0e534-7ff0-49f8-946e-9df46dca5ddd admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:33.694060 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f5d0e534-7ff0-49f8-946e-9df46dca5ddd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71d1c4930a1b487eb3dca8fde6ce424a', 'request_id': 'req-f5d0e534-7ff0-49f8-946e-9df46dca5ddd'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:33.695930 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:43:33.701004 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f5d0e534-7ff0-49f8-946e-9df46dca5ddd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71d1c4930a1b487eb3dca8fde6ce424a', 'request_id': 'req-f5d0e534-7ff0-49f8-946e-9df46dca5ddd'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:33.706807 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f5d0e534-7ff0-49f8-946e-9df46dca5ddd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71d1c4930a1b487eb3dca8fde6ce424a', 'request_id': 'req-f5d0e534-7ff0-49f8-946e-9df46dca5ddd'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:33.718562 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f5d0e534-7ff0-49f8-946e-9df46dca5ddd admin admin] The token cache is being invalidate because project 71d1c4930a1b487eb3dca8fde6ce424a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:43:33.718844 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f5d0e534-7ff0-49f8-946e-9df46dca5ddd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:33.719452 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f5d0e534-7ff0-49f8-946e-9df46dca5ddd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:33.720958 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1060/2128] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:43:33 2022] DELETE /identity/v3/projects/71d1c4930a1b487eb3dca8fde6ce424a => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:43:33.726400 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b1646-f418-40a6-99fc-e885dca80e12 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:33.726687 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b1646-f418-40a6-99fc-e885dca80e12 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:33.726982 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b1646-f418-40a6-99fc-e885dca80e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:33.747070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-177b1646-f418-40a6-99fc-e885dca80e12 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:43:33.748257 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-177b1646-f418-40a6-99fc-e885dca80e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:33.804354 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1069/2129] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:43:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:43:33.813229 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7c0a2-be27-4b50-aa55-2739b783983f None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:33.813506 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7c0a2-be27-4b50-aa55-2739b783983f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:33.813781 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7c0a2-be27-4b50-aa55-2739b783983f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:33.833716 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b6a7c0a2-be27-4b50-aa55-2739b783983f None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:43:33.834922 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b6a7c0a2-be27-4b50-aa55-2739b783983f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:33.843137 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1061/2130] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:43:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:43:33.847152 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177b1646-f418-40a6-99fc-e885dca80e12 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:33.857213 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177b1646-f418-40a6-99fc-e885dca80e12 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:33.861752 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd124bbb-7e9f-43b3-96f6-63b77eb09dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:33.862345 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd124bbb-7e9f-43b3-96f6-63b77eb09dc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:33.862623 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd124bbb-7e9f-43b3-96f6-63b77eb09dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:33.862907 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd124bbb-7e9f-43b3-96f6-63b77eb09dc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:33.863536 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd124bbb-7e9f-43b3-96f6-63b77eb09dc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:43:33.863837 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd124bbb-7e9f-43b3-96f6-63b77eb09dc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:33.865483 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd124bbb-7e9f-43b3-96f6-63b77eb09dc1 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:33.877385 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1070/2131] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:43:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:43:33.881519 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a7c0a2-be27-4b50-aa55-2739b783983f None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:33.890781 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a7c0a2-be27-4b50-aa55-2739b783983f None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:33.895382 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132ac8c4-8255-4907-a014-953c0708284a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:33.895964 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132ac8c4-8255-4907-a014-953c0708284a admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:33.896235 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132ac8c4-8255-4907-a014-953c0708284a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:33.896498 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132ac8c4-8255-4907-a014-953c0708284a admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:33.897174 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132ac8c4-8255-4907-a014-953c0708284a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:33.899067 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132ac8c4-8255-4907-a014-953c0708284a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:33.928963 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1062/2132] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:43:33 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:43:33.930580 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd124bbb-7e9f-43b3-96f6-63b77eb09dc1 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:33.941325 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd124bbb-7e9f-43b3-96f6-63b77eb09dc1 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:34.005022 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55239391-eaa0-4c03-a129-3436d73ff723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:34.005662 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55239391-eaa0-4c03-a129-3436d73ff723 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:34.005939 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55239391-eaa0-4c03-a129-3436d73ff723 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:34.006209 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55239391-eaa0-4c03-a129-3436d73ff723 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:34.006849 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55239391-eaa0-4c03-a129-3436d73ff723 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:34.008784 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55239391-eaa0-4c03-a129-3436d73ff723 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:34.017014 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:43:34.032988 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1071/2133] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:43:33 2022] POST /identity/v3/users => generated 286 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:43:34.037047 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132ac8c4-8255-4907-a014-953c0708284a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:34.048152 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132ac8c4-8255-4907-a014-953c0708284a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:34.054303 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b2f64-dc01-4652-a7cb-68840620d451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:34.054978 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b2f64-dc01-4652-a7cb-68840620d451 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:34.055193 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b2f64-dc01-4652-a7cb-68840620d451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:34.055472 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b2f64-dc01-4652-a7cb-68840620d451 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:34.056112 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3b2f64-dc01-4652-a7cb-68840620d451 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:34.057765 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3b2f64-dc01-4652-a7cb-68840620d451 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:34.068225 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1063/2134] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:43:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:43:34.072240 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55239391-eaa0-4c03-a129-3436d73ff723 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:34.082607 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55239391-eaa0-4c03-a129-3436d73ff723 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:34.087463 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a1e9ed-e1f5-4edd-8f7b-c602e2719c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:34.088636 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1e9ed-e1f5-4edd-8f7b-c602e2719c8a admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:34.088913 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1e9ed-e1f5-4edd-8f7b-c602e2719c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:34.089179 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1e9ed-e1f5-4edd-8f7b-c602e2719c8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:34.089806 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a1e9ed-e1f5-4edd-8f7b-c602e2719c8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:34.091394 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a1e9ed-e1f5-4edd-8f7b-c602e2719c8a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:34.101944 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1072/2135] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:43:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:43:34.106491 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b2f64-dc01-4652-a7cb-68840620d451 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:34.116609 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b2f64-dc01-4652-a7cb-68840620d451 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:34.121571 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d623a9-33b9-479c-baec-dfa4cb651348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:34.122209 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d623a9-33b9-479c-baec-dfa4cb651348 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:34.122487 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d623a9-33b9-479c-baec-dfa4cb651348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:34.122761 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d623a9-33b9-479c-baec-dfa4cb651348 admin admin] PATH_INFO: `/v3/projects/6e307e105e324d5ea679bb563088b142/users/aebd611119b840a0b8ebfb8d14055cfb/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:34.137145 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d623a9-33b9-479c-baec-dfa4cb651348 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e307e105e324d5ea679bb563088b142, user_id=aebd611119b840a0b8ebfb8d14055cfb, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:34.139794 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d623a9-33b9-479c-baec-dfa4cb651348 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:34.153015 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1064/2136] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:43:34 2022] PUT /identity/v3/projects/6e307e105e324d5ea679bb563088b142/users/aebd611119b840a0b8ebfb8d14055cfb/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:43:34.163168 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd8520-e416-4a72-a623-f55ae3bbfe12 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:34.163476 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd8520-e416-4a72-a623-f55ae3bbfe12 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:34.163745 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd8520-e416-4a72-a623-f55ae3bbfe12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:34.191022 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-05cd8520-e416-4a72-a623-f55ae3bbfe12 None None] MFA Rules not processed for user `aebd611119b840a0b8ebfb8d14055cfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:43:34.192401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-05cd8520-e416-4a72-a623-f55ae3bbfe12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:34.254710 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1073/2137] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:43:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:43:34.265807 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d623a9-33b9-479c-baec-dfa4cb651348 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:34.267610 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a0d623a9-33b9-479c-baec-dfa4cb651348 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:34.345108 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d623a9-33b9-479c-baec-dfa4cb651348 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:34.350904 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e5eb16-98f8-4694-9b37-4df7b7c0ce7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe4a6e4b6184826b30ac4fc13ad9b1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:34.351506 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e5eb16-98f8-4694-9b37-4df7b7c0ce7d service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:34.351787 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e5eb16-98f8-4694-9b37-4df7b7c0ce7d service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:34.352057 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e5eb16-98f8-4694-9b37-4df7b7c0ce7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:34.360075 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e5eb16-98f8-4694-9b37-4df7b7c0ce7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:34.361692 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e5eb16-98f8-4694-9b37-4df7b7c0ce7d service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:34.371067 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1065/2138] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:43:34 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:43:34.537273 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e97b8-69fd-44d8-aa57-a85049ba4759 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:34.537704 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e97b8-69fd-44d8-aa57-a85049ba4759 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:34.538107 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e97b8-69fd-44d8-aa57-a85049ba4759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:34.569274 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-3c8e97b8-69fd-44d8-aa57-a85049ba4759 None None] MFA Rules not processed for user `7f389510a6a044d8a7487561b169544b`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:43:34.584542 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3c8e97b8-69fd-44d8-aa57-a85049ba4759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:34.635172 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1074/2139] 10.222.0.70 () {58 vars in 967 bytes} [Thu Oct 27 09:43:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:43:34.658448 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e5eb16-98f8-4694-9b37-4df7b7c0ce7d service nova] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:34.659864 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-52e5eb16-98f8-4694-9b37-4df7b7c0ce7d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:34.732585 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:43:34.740962 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e5eb16-98f8-4694-9b37-4df7b7c0ce7d service nova] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:34.747754 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b303f186-6f8d-40e1-b731-0d892827b276 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eecf79ea4700496c89b1a67c614b34d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:34.748536 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b303f186-6f8d-40e1-b731-0d892827b276 service swift] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:34.748649 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b303f186-6f8d-40e1-b731-0d892827b276 service swift] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:34.748914 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b303f186-6f8d-40e1-b731-0d892827b276 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:34.757366 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b303f186-6f8d-40e1-b731-0d892827b276 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:34.758906 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b303f186-6f8d-40e1-b731-0d892827b276 service swift] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:34.768266 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1066/2140] 10.222.0.70 () {60 vars in 1384 bytes} [Thu Oct 27 09:43:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 27 09:43:37.118866 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e97b8-69fd-44d8-aa57-a85049ba4759 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:37.178791 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e97b8-69fd-44d8-aa57-a85049ba4759 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:37.183907 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e49958e-dfa3-4da6-aea0-67f941031d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:37.184536 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e49958e-dfa3-4da6-aea0-67f941031d27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:37.184817 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e49958e-dfa3-4da6-aea0-67f941031d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:37.185078 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e49958e-dfa3-4da6-aea0-67f941031d27 admin admin] PATH_INFO: `/v3/users/aebd611119b840a0b8ebfb8d14055cfb` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:37.186159 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e49958e-dfa3-4da6-aea0-67f941031d27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aebd611119b840a0b8ebfb8d14055cfb)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:37.188080 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e49958e-dfa3-4da6-aea0-67f941031d27 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:37.201405 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:INSERT=3,keystone:SELECT=96,keystone:DELETE=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:43:37.234285 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9e49958e-dfa3-4da6-aea0-67f941031d27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aebd611119b840a0b8ebfb8d14055cfb'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:37.241127 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9e49958e-dfa3-4da6-aea0-67f941031d27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aebd611119b840a0b8ebfb8d14055cfb'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:37.253862 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9e49958e-dfa3-4da6-aea0-67f941031d27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aebd611119b840a0b8ebfb8d14055cfb'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:37.260923 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9e49958e-dfa3-4da6-aea0-67f941031d27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aebd611119b840a0b8ebfb8d14055cfb'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:37.261528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9e49958e-dfa3-4da6-aea0-67f941031d27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aebd611119b840a0b8ebfb8d14055cfb'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:37.267311 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1075/2141] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:43:37 2022] DELETE /identity/v3/users/aebd611119b840a0b8ebfb8d14055cfb => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:43:37.271979 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641e73b-51e6-4ff7-9707-3ef22121ae7b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:37.272472 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641e73b-51e6-4ff7-9707-3ef22121ae7b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:37.272637 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641e73b-51e6-4ff7-9707-3ef22121ae7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:37.292202 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b641e73b-51e6-4ff7-9707-3ef22121ae7b None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:43:37.293441 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b641e73b-51e6-4ff7-9707-3ef22121ae7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:37.351596 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1067/2142] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:43:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:43:37.361716 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e49958e-dfa3-4da6-aea0-67f941031d27 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:37.363098 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9e49958e-dfa3-4da6-aea0-67f941031d27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:37.419432 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e49958e-dfa3-4da6-aea0-67f941031d27 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:37.423011 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a4a68-acf4-4abd-a881-addd978fedd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:37.423608 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a4a68-acf4-4abd-a881-addd978fedd5 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:37.423889 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a4a68-acf4-4abd-a881-addd978fedd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:37.424160 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a4a68-acf4-4abd-a881-addd978fedd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:37.433049 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0a4a68-acf4-4abd-a881-addd978fedd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:37.434460 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0a4a68-acf4-4abd-a881-addd978fedd5 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:37.443744 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1076/2143] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:43:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:43:37.704428 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641e73b-51e6-4ff7-9707-3ef22121ae7b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:37.706264 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b641e73b-51e6-4ff7-9707-3ef22121ae7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:37.710100 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a4a68-acf4-4abd-a881-addd978fedd5 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:37.711557 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2e0a4a68-acf4-4abd-a881-addd978fedd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:37.719029 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641e73b-51e6-4ff7-9707-3ef22121ae7b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:37.725710 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a4a68-acf4-4abd-a881-addd978fedd5 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:37.730256 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4e3de6-1e88-42d1-bfb7-9e9f434c426e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:37.730886 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e3de6-1e88-42d1-bfb7-9e9f434c426e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:37.731164 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e3de6-1e88-42d1-bfb7-9e9f434c426e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:37.731434 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e3de6-1e88-42d1-bfb7-9e9f434c426e admin admin] PATH_INFO: `/v3/users/70b618ffc3044d919391a60ca2757a3c` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:37.732577 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4e3de6-1e88-42d1-bfb7-9e9f434c426e admin admin] RBAC: Authorizing `identity:delete_user(user_id=70b618ffc3044d919391a60ca2757a3c)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:37.734152 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37177f9-50f6-4dd5-b56e-8415bbe0bff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:37.734507 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4e3de6-1e88-42d1-bfb7-9e9f434c426e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:37.735388 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37177f9-50f6-4dd5-b56e-8415bbe0bff4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:37.735772 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37177f9-50f6-4dd5-b56e-8415bbe0bff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:37.736609 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37177f9-50f6-4dd5-b56e-8415bbe0bff4 admin admin] PATH_INFO: `/v3/projects/6e307e105e324d5ea679bb563088b142` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:37.738190 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37177f9-50f6-4dd5-b56e-8415bbe0bff4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e307e105e324d5ea679bb563088b142)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:37.740030 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37177f9-50f6-4dd5-b56e-8415bbe0bff4 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:37.784498 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ee4e3de6-1e88-42d1-bfb7-9e9f434c426e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70b618ffc3044d919391a60ca2757a3c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:37.789245 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a37177f9-50f6-4dd5-b56e-8415bbe0bff4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e307e105e324d5ea679bb563088b142', 'request_id': 'req-a37177f9-50f6-4dd5-b56e-8415bbe0bff4'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:37.792142 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ee4e3de6-1e88-42d1-bfb7-9e9f434c426e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70b618ffc3044d919391a60ca2757a3c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:37.796280 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a37177f9-50f6-4dd5-b56e-8415bbe0bff4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e307e105e324d5ea679bb563088b142', 'request_id': 'req-a37177f9-50f6-4dd5-b56e-8415bbe0bff4'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:37.802233 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a37177f9-50f6-4dd5-b56e-8415bbe0bff4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e307e105e324d5ea679bb563088b142', 'request_id': 'req-a37177f9-50f6-4dd5-b56e-8415bbe0bff4'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:37.804431 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ee4e3de6-1e88-42d1-bfb7-9e9f434c426e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70b618ffc3044d919391a60ca2757a3c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:37.811987 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ee4e3de6-1e88-42d1-bfb7-9e9f434c426e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70b618ffc3044d919391a60ca2757a3c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:37.812642 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ee4e3de6-1e88-42d1-bfb7-9e9f434c426e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70b618ffc3044d919391a60ca2757a3c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:37.814655 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a37177f9-50f6-4dd5-b56e-8415bbe0bff4 admin admin] The token cache is being invalidate because project 6e307e105e324d5ea679bb563088b142 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:43:37.814944 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a37177f9-50f6-4dd5-b56e-8415bbe0bff4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:37.815563 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a37177f9-50f6-4dd5-b56e-8415bbe0bff4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:37.817133 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1068/2144] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:43:37 2022] DELETE /identity/v3/projects/6e307e105e324d5ea679bb563088b142 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:43:37.818827 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1077/2145] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:43:37 2022] DELETE /identity/v3/users/70b618ffc3044d919391a60ca2757a3c => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:43:37.821738 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd75b5-f03b-433c-aa4e-95078466b306 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:37.822023 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd75b5-f03b-433c-aa4e-95078466b306 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:37.822830 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd75b5-f03b-433c-aa4e-95078466b306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:37.823716 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d3d18-5002-433c-ba2f-c78d9eff17e2 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:37.824012 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d3d18-5002-433c-ba2f-c78d9eff17e2 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:37.824268 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d3d18-5002-433c-ba2f-c78d9eff17e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:37.837151 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:43:37.843094 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-4edd75b5-f03b-433c-aa4e-95078466b306 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:43:37.844304 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4edd75b5-f03b-433c-aa4e-95078466b306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:37.844662 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-014d3d18-5002-433c-ba2f-c78d9eff17e2 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:43:37.846627 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-014d3d18-5002-433c-ba2f-c78d9eff17e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:37.907430 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1069/2146] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:43:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:43:37.911627 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1078/2147] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:43:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:43:37.915068 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720990b7-4213-40c0-ab07-aa38e192bc01 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:37.915356 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720990b7-4213-40c0-ab07-aa38e192bc01 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:37.915629 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720990b7-4213-40c0-ab07-aa38e192bc01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:37.918575 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d3d18-5002-433c-ba2f-c78d9eff17e2 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:37.919973 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-014d3d18-5002-433c-ba2f-c78d9eff17e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:37.935649 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-720990b7-4213-40c0-ab07-aa38e192bc01 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:43:37.936874 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-720990b7-4213-40c0-ab07-aa38e192bc01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:37.944929 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1070/2148] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:43:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:43:37.949922 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720990b7-4213-40c0-ab07-aa38e192bc01 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:37.959764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720990b7-4213-40c0-ab07-aa38e192bc01 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:37.965322 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16347c77-f461-406f-9791-7c93dae75c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:37.965886 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16347c77-f461-406f-9791-7c93dae75c6a admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:37.966170 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16347c77-f461-406f-9791-7c93dae75c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:37.966441 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16347c77-f461-406f-9791-7c93dae75c6a admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:37.967073 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16347c77-f461-406f-9791-7c93dae75c6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:43:37.967375 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16347c77-f461-406f-9791-7c93dae75c6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:37.969001 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16347c77-f461-406f-9791-7c93dae75c6a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:37.971402 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:43:37.979107 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d3d18-5002-433c-ba2f-c78d9eff17e2 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:37.983271 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c3bf4c-92d1-41f8-9dbd-380d68a73429 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:37.983880 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3bf4c-92d1-41f8-9dbd-380d68a73429 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:37.984161 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3bf4c-92d1-41f8-9dbd-380d68a73429 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:37.984435 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3bf4c-92d1-41f8-9dbd-380d68a73429 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:37.985092 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1071/2149] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:43:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:43:37.985931 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16347c77-f461-406f-9791-7c93dae75c6a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:37.994128 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c3bf4c-92d1-41f8-9dbd-380d68a73429 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:37.995552 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c3bf4c-92d1-41f8-9dbd-380d68a73429 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:37.995853 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16347c77-f461-406f-9791-7c93dae75c6a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:38.000627 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868c28d4-164f-4c5c-97b0-ea7a7bde43b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:38.001206 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c28d4-164f-4c5c-97b0-ea7a7bde43b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:38.001481 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c28d4-164f-4c5c-97b0-ea7a7bde43b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:38.001752 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c28d4-164f-4c5c-97b0-ea7a7bde43b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:38.002367 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868c28d4-164f-4c5c-97b0-ea7a7bde43b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:38.004426 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868c28d4-164f-4c5c-97b0-ea7a7bde43b2 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:38.005079 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1079/2150] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:43:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 09:43:38.036556 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1072/2151] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:43:37 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:43:38.037980 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c3bf4c-92d1-41f8-9dbd-380d68a73429 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:38.048151 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c3bf4c-92d1-41f8-9dbd-380d68a73429 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:38.108639 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95060e7-595c-49f1-8eee-5a6944bd98e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:38.109261 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95060e7-595c-49f1-8eee-5a6944bd98e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:38.109537 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95060e7-595c-49f1-8eee-5a6944bd98e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:38.109808 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95060e7-595c-49f1-8eee-5a6944bd98e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:38.110436 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95060e7-595c-49f1-8eee-5a6944bd98e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:38.112350 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95060e7-595c-49f1-8eee-5a6944bd98e3 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:38.135688 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1080/2152] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:43:38 2022] POST /identity/v3/users => generated 284 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:43:38.139826 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868c28d4-164f-4c5c-97b0-ea7a7bde43b2 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:38.150421 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868c28d4-164f-4c5c-97b0-ea7a7bde43b2 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:38.159855 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b9d44f-ef78-4d1e-bcba-a49ecee65cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:38.160900 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b9d44f-ef78-4d1e-bcba-a49ecee65cba admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:38.160900 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b9d44f-ef78-4d1e-bcba-a49ecee65cba admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:38.161133 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b9d44f-ef78-4d1e-bcba-a49ecee65cba admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:38.161718 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b9d44f-ef78-4d1e-bcba-a49ecee65cba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:38.163352 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b9d44f-ef78-4d1e-bcba-a49ecee65cba admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:38.174093 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1073/2153] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:43:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:43:38.178015 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95060e7-595c-49f1-8eee-5a6944bd98e3 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:38.188343 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95060e7-595c-49f1-8eee-5a6944bd98e3 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:38.193589 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b3185-0f53-4fef-b6cc-3e4d89e0acca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:38.194194 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b3185-0f53-4fef-b6cc-3e4d89e0acca admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:38.194469 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b3185-0f53-4fef-b6cc-3e4d89e0acca admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:38.194731 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b3185-0f53-4fef-b6cc-3e4d89e0acca admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:38.195357 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7b3185-0f53-4fef-b6cc-3e4d89e0acca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:38.196969 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7b3185-0f53-4fef-b6cc-3e4d89e0acca admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:38.208886 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1081/2154] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:43:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:43:38.213107 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b9d44f-ef78-4d1e-bcba-a49ecee65cba admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:38.223317 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b9d44f-ef78-4d1e-bcba-a49ecee65cba admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:38.228063 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ec55e7-f95b-4346-a72c-f11ab055a2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:38.228800 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec55e7-f95b-4346-a72c-f11ab055a2f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:38.229077 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec55e7-f95b-4346-a72c-f11ab055a2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:38.229348 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec55e7-f95b-4346-a72c-f11ab055a2f4 admin admin] PATH_INFO: `/v3/projects/cf2bcb07297f478aa3dffe15bc931d53/users/5e2f89388c744b4cb8f291bd04c63d60/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:38.245123 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ec55e7-f95b-4346-a72c-f11ab055a2f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf2bcb07297f478aa3dffe15bc931d53, user_id=5e2f89388c744b4cb8f291bd04c63d60, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:38.247778 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ec55e7-f95b-4346-a72c-f11ab055a2f4 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:38.261561 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1074/2155] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:43:38 2022] PUT /identity/v3/projects/cf2bcb07297f478aa3dffe15bc931d53/users/5e2f89388c744b4cb8f291bd04c63d60/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:43:38.272884 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db71cc41-5bf9-4f04-bc0e-4116898d9af5 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:38.273171 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db71cc41-5bf9-4f04-bc0e-4116898d9af5 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:38.273497 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db71cc41-5bf9-4f04-bc0e-4116898d9af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:38.286682 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ec55e7-f95b-4346-a72c-f11ab055a2f4 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:38.288652 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a6ec55e7-f95b-4346-a72c-f11ab055a2f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:38.310923 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-db71cc41-5bf9-4f04-bc0e-4116898d9af5 None None] MFA Rules not processed for user `5e2f89388c744b4cb8f291bd04c63d60`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:43:38.312156 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-db71cc41-5bf9-4f04-bc0e-4116898d9af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:38.330365 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:43:38.360250 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ec55e7-f95b-4346-a72c-f11ab055a2f4 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:38.365004 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec711d0d-a7ff-45dd-ac5c-8755f0d7f07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:38.365632 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec711d0d-a7ff-45dd-ac5c-8755f0d7f07c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:38.365910 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec711d0d-a7ff-45dd-ac5c-8755f0d7f07c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:38.366185 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec711d0d-a7ff-45dd-ac5c-8755f0d7f07c admin admin] PATH_INFO: `/v3/projects/80470dda88894ca282a2fe303df96ef3` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:38.367315 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec711d0d-a7ff-45dd-ac5c-8755f0d7f07c admin admin] RBAC: Authorizing `identity:delete_project(project_id=80470dda88894ca282a2fe303df96ef3)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:38.369816 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec711d0d-a7ff-45dd-ac5c-8755f0d7f07c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:38.382953 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1082/2156] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:43:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:43:38.394217 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db71cc41-5bf9-4f04-bc0e-4116898d9af5 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:38.395591 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-db71cc41-5bf9-4f04-bc0e-4116898d9af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:38.420922 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ec711d0d-a7ff-45dd-ac5c-8755f0d7f07c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80470dda88894ca282a2fe303df96ef3', 'request_id': 'req-ec711d0d-a7ff-45dd-ac5c-8755f0d7f07c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:38.427723 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ec711d0d-a7ff-45dd-ac5c-8755f0d7f07c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80470dda88894ca282a2fe303df96ef3', 'request_id': 'req-ec711d0d-a7ff-45dd-ac5c-8755f0d7f07c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:38.434076 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ec711d0d-a7ff-45dd-ac5c-8755f0d7f07c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80470dda88894ca282a2fe303df96ef3', 'request_id': 'req-ec711d0d-a7ff-45dd-ac5c-8755f0d7f07c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:38.446939 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ec711d0d-a7ff-45dd-ac5c-8755f0d7f07c admin admin] The token cache is being invalidate because project 80470dda88894ca282a2fe303df96ef3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:43:38.447823 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ec711d0d-a7ff-45dd-ac5c-8755f0d7f07c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:38.448766 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ec711d0d-a7ff-45dd-ac5c-8755f0d7f07c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:38.453661 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1075/2157] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:43:38 2022] DELETE /identity/v3/projects/80470dda88894ca282a2fe303df96ef3 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:43:38.454996 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9c9c0-b7e8-4dd6-9e3a-cae20b03734e None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:38.455278 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9c9c0-b7e8-4dd6-9e3a-cae20b03734e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:38.455548 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9c9c0-b7e8-4dd6-9e3a-cae20b03734e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:38.456449 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db71cc41-5bf9-4f04-bc0e-4116898d9af5 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:38.475094 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-3aa9c9c0-b7e8-4dd6-9e3a-cae20b03734e None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:43:38.476625 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3aa9c9c0-b7e8-4dd6-9e3a-cae20b03734e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:38.501807 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:43:38.504639 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce49972e-f1a7-4700-9121-fe9b9f693c4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:38.505244 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce49972e-f1a7-4700-9121-fe9b9f693c4c service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:38.505526 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce49972e-f1a7-4700-9121-fe9b9f693c4c service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:38.505796 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce49972e-f1a7-4700-9121-fe9b9f693c4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:38.507760 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ce49972e-f1a7-4700-9121-fe9b9f693c4c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:38.539424 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1076/2158] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:43:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:43:38.548814 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5950f-9313-4b2d-bd55-5e84f4c91fcf None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:38.549098 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5950f-9313-4b2d-bd55-5e84f4c91fcf None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:38.549387 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5950f-9313-4b2d-bd55-5e84f4c91fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:38.570278 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-68a5950f-9313-4b2d-bd55-5e84f4c91fcf None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:43:38.570984 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce49972e-f1a7-4700-9121-fe9b9f693c4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:38.571474 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-68a5950f-9313-4b2d-bd55-5e84f4c91fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:38.572445 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce49972e-f1a7-4700-9121-fe9b9f693c4c service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:38.579984 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1077/2159] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:43:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:43:38.581968 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1083/2160] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:43:38 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 09:43:38.583952 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a5950f-9313-4b2d-bd55-5e84f4c91fcf None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:38.593212 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a5950f-9313-4b2d-bd55-5e84f4c91fcf None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:38.598978 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2290b8-0519-40be-8e87-95b1dd818b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:38.599586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2290b8-0519-40be-8e87-95b1dd818b13 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:38.599860 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2290b8-0519-40be-8e87-95b1dd818b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:38.600130 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2290b8-0519-40be-8e87-95b1dd818b13 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:38.600806 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2290b8-0519-40be-8e87-95b1dd818b13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:43:38.601107 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2290b8-0519-40be-8e87-95b1dd818b13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:38.602735 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2290b8-0519-40be-8e87-95b1dd818b13 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:38.615403 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1078/2161] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:43:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:43:38.620347 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce49972e-f1a7-4700-9121-fe9b9f693c4c service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:38.630820 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce49972e-f1a7-4700-9121-fe9b9f693c4c service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:38.635766 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3954b79-bbd0-44d5-9cd5-e9bb91f8884d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:38.636539 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3954b79-bbd0-44d5-9cd5-e9bb91f8884d admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:38.637511 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3954b79-bbd0-44d5-9cd5-e9bb91f8884d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:38.637796 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3954b79-bbd0-44d5-9cd5-e9bb91f8884d admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:38.638446 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3954b79-bbd0-44d5-9cd5-e9bb91f8884d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:38.640334 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3954b79-bbd0-44d5-9cd5-e9bb91f8884d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:38.670670 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1084/2162] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:43:38 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:43:38.671869 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2290b8-0519-40be-8e87-95b1dd818b13 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:38.682336 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2290b8-0519-40be-8e87-95b1dd818b13 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:38.741594 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f39ce-2503-450f-b5f1-5094abca3869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:38.742205 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f39ce-2503-450f-b5f1-5094abca3869 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:38.742615 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f39ce-2503-450f-b5f1-5094abca3869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:38.742956 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f39ce-2503-450f-b5f1-5094abca3869 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:38.743657 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03f39ce-2503-450f-b5f1-5094abca3869 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:38.746202 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03f39ce-2503-450f-b5f1-5094abca3869 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:38.767489 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1079/2163] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:43:38 2022] POST /identity/v3/users => generated 285 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:43:38.771255 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3954b79-bbd0-44d5-9cd5-e9bb91f8884d admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:38.781470 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3954b79-bbd0-44d5-9cd5-e9bb91f8884d admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:38.786886 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5802cf8-060c-4add-ab5d-9e5d224843bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:38.788236 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5802cf8-060c-4add-ab5d-9e5d224843bb admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:38.788545 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5802cf8-060c-4add-ab5d-9e5d224843bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:38.789003 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5802cf8-060c-4add-ab5d-9e5d224843bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:38.789611 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5802cf8-060c-4add-ab5d-9e5d224843bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:38.791777 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5802cf8-060c-4add-ab5d-9e5d224843bb admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:38.793167 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:43:38.804416 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1085/2164] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:43:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:43:38.808859 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f39ce-2503-450f-b5f1-5094abca3869 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:38.820542 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f39ce-2503-450f-b5f1-5094abca3869 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:38.825614 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78f8962-58b7-4644-8bc2-d3d3faebc09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:38.826219 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f8962-58b7-4644-8bc2-d3d3faebc09a admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:38.826497 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f8962-58b7-4644-8bc2-d3d3faebc09a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:38.826799 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f8962-58b7-4644-8bc2-d3d3faebc09a admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:38.827433 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78f8962-58b7-4644-8bc2-d3d3faebc09a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:38.829102 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78f8962-58b7-4644-8bc2-d3d3faebc09a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:38.839990 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1080/2165] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:43:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:43:38.844271 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5802cf8-060c-4add-ab5d-9e5d224843bb admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:38.854940 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5802cf8-060c-4add-ab5d-9e5d224843bb admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:38.860364 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf6120-b757-400c-aad7-8f081d7c47c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:38.860705 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf6120-b757-400c-aad7-8f081d7c47c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:38.860834 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf6120-b757-400c-aad7-8f081d7c47c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:38.861105 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf6120-b757-400c-aad7-8f081d7c47c2 admin admin] PATH_INFO: `/v3/projects/f44a958112694aab911d7acaa525ed95/users/5fb3071146d94057a652689804588e53/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:38.879253 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baf6120-b757-400c-aad7-8f081d7c47c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f44a958112694aab911d7acaa525ed95, user_id=5fb3071146d94057a652689804588e53, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:38.882058 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baf6120-b757-400c-aad7-8f081d7c47c2 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:38.898018 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1086/2166] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:43:38 2022] PUT /identity/v3/projects/f44a958112694aab911d7acaa525ed95/users/5fb3071146d94057a652689804588e53/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:43:38.908627 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4886db67-9541-4fb0-a275-72eb25b834e7 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:38.908916 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4886db67-9541-4fb0-a275-72eb25b834e7 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:38.909191 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4886db67-9541-4fb0-a275-72eb25b834e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:38.938620 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-4886db67-9541-4fb0-a275-72eb25b834e7 None None] MFA Rules not processed for user `5fb3071146d94057a652689804588e53`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:43:38.939832 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4886db67-9541-4fb0-a275-72eb25b834e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:39.004730 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1081/2167] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:43:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:43:39.016319 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf6120-b757-400c-aad7-8f081d7c47c2 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:39.018049 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8baf6120-b757-400c-aad7-8f081d7c47c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:39.073677 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf6120-b757-400c-aad7-8f081d7c47c2 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:39.077289 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e7ebcd-7528-47a0-8445-6d90f415fd22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:39.077883 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7ebcd-7528-47a0-8445-6d90f415fd22 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:39.078159 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7ebcd-7528-47a0-8445-6d90f415fd22 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:39.078429 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7ebcd-7528-47a0-8445-6d90f415fd22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:39.086476 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e7ebcd-7528-47a0-8445-6d90f415fd22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:39.088371 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e7ebcd-7528-47a0-8445-6d90f415fd22 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:39.096866 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1087/2168] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:43:39 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:43:39.123236 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4886db67-9541-4fb0-a275-72eb25b834e7 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:39.142402 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:43:39.186304 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4886db67-9541-4fb0-a275-72eb25b834e7 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:39.190980 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f52ef72-484a-4f36-b9a0-c850a064d89b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:39.191545 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f52ef72-484a-4f36-b9a0-c850a064d89b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:39.191824 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f52ef72-484a-4f36-b9a0-c850a064d89b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:39.192095 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f52ef72-484a-4f36-b9a0-c850a064d89b admin admin] PATH_INFO: `/v3/users/5fb3071146d94057a652689804588e53` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:39.193238 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f52ef72-484a-4f36-b9a0-c850a064d89b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fb3071146d94057a652689804588e53)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:39.195162 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f52ef72-484a-4f36-b9a0-c850a064d89b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:39.239304 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0f52ef72-484a-4f36-b9a0-c850a064d89b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fb3071146d94057a652689804588e53'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:39.246888 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0f52ef72-484a-4f36-b9a0-c850a064d89b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fb3071146d94057a652689804588e53'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:39.258499 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0f52ef72-484a-4f36-b9a0-c850a064d89b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fb3071146d94057a652689804588e53'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:39.264947 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0f52ef72-484a-4f36-b9a0-c850a064d89b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fb3071146d94057a652689804588e53'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:39.265648 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0f52ef72-484a-4f36-b9a0-c850a064d89b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fb3071146d94057a652689804588e53'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:39.271465 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1082/2169] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:43:39 2022] DELETE /identity/v3/users/5fb3071146d94057a652689804588e53 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:43:39.275980 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1119d22-ff6e-45b3-b4c6-c4a054cdce5e None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:39.276492 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1119d22-ff6e-45b3-b4c6-c4a054cdce5e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:39.276614 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1119d22-ff6e-45b3-b4c6-c4a054cdce5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:39.296006 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-b1119d22-ff6e-45b3-b4c6-c4a054cdce5e None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:43:39.297249 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b1119d22-ff6e-45b3-b4c6-c4a054cdce5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:39.356000 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1088/2170] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:43:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:43:39.365819 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f52ef72-484a-4f36-b9a0-c850a064d89b admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:39.367214 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0f52ef72-484a-4f36-b9a0-c850a064d89b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:39.425159 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f52ef72-484a-4f36-b9a0-c850a064d89b admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:39.428815 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f715a4f-6946-4e19-bae3-6c74230452e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:39.429413 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f715a4f-6946-4e19-bae3-6c74230452e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:39.429685 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f715a4f-6946-4e19-bae3-6c74230452e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:39.429949 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f715a4f-6946-4e19-bae3-6c74230452e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:39.438711 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f715a4f-6946-4e19-bae3-6c74230452e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:39.440133 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f715a4f-6946-4e19-bae3-6c74230452e7 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:39.450224 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1083/2171] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:43:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:43:39.720133 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1119d22-ff6e-45b3-b4c6-c4a054cdce5e None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:39.721582 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b1119d22-ff6e-45b3-b4c6-c4a054cdce5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:39.734013 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1119d22-ff6e-45b3-b4c6-c4a054cdce5e None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:39.738774 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da61279-2b95-44c2-8260-951e1f3c6582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:39.739479 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da61279-2b95-44c2-8260-951e1f3c6582 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:39.739888 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da61279-2b95-44c2-8260-951e1f3c6582 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:39.740252 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da61279-2b95-44c2-8260-951e1f3c6582 admin admin] PATH_INFO: `/v3/projects/f44a958112694aab911d7acaa525ed95` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:39.741590 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da61279-2b95-44c2-8260-951e1f3c6582 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f44a958112694aab911d7acaa525ed95)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:39.743372 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da61279-2b95-44c2-8260-951e1f3c6582 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:39.789844 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2da61279-2b95-44c2-8260-951e1f3c6582 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f44a958112694aab911d7acaa525ed95', 'request_id': 'req-2da61279-2b95-44c2-8260-951e1f3c6582'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:39.795441 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2da61279-2b95-44c2-8260-951e1f3c6582 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f44a958112694aab911d7acaa525ed95', 'request_id': 'req-2da61279-2b95-44c2-8260-951e1f3c6582'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:39.801789 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2da61279-2b95-44c2-8260-951e1f3c6582 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f44a958112694aab911d7acaa525ed95', 'request_id': 'req-2da61279-2b95-44c2-8260-951e1f3c6582'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:39.809559 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:43:39.814903 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2da61279-2b95-44c2-8260-951e1f3c6582 admin admin] The token cache is being invalidate because project f44a958112694aab911d7acaa525ed95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:43:39.815312 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2da61279-2b95-44c2-8260-951e1f3c6582 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:39.816796 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2da61279-2b95-44c2-8260-951e1f3c6582 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:39.818410 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1089/2172] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:43:39 2022] DELETE /identity/v3/projects/f44a958112694aab911d7acaa525ed95 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:43:39.824109 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90c942-136d-46b1-a706-68d15917df75 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:39.824387 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90c942-136d-46b1-a706-68d15917df75 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:39.824699 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90c942-136d-46b1-a706-68d15917df75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:39.843458 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-7b90c942-136d-46b1-a706-68d15917df75 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:43:39.844673 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7b90c942-136d-46b1-a706-68d15917df75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:39.877644 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:43:39.904328 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1084/2173] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:43:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:43:39.912074 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372971bb-b89e-4618-8683-86ec8f938269 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:39.912342 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372971bb-b89e-4618-8683-86ec8f938269 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:39.912637 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372971bb-b89e-4618-8683-86ec8f938269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:39.933794 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-372971bb-b89e-4618-8683-86ec8f938269 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:43:39.935024 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-372971bb-b89e-4618-8683-86ec8f938269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:39.943359 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1090/2174] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:43:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:43:39.948116 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b90c942-136d-46b1-a706-68d15917df75 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:39.958296 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b90c942-136d-46b1-a706-68d15917df75 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:39.962932 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34230bf3-4997-48e4-8f81-88a0cafceb18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:39.963465 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34230bf3-4997-48e4-8f81-88a0cafceb18 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:39.963740 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34230bf3-4997-48e4-8f81-88a0cafceb18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:39.964009 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34230bf3-4997-48e4-8f81-88a0cafceb18 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:39.964681 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34230bf3-4997-48e4-8f81-88a0cafceb18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:43:39.965012 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34230bf3-4997-48e4-8f81-88a0cafceb18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:39.966594 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34230bf3-4997-48e4-8f81-88a0cafceb18 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:39.978901 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1085/2175] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:43:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:43:39.983128 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372971bb-b89e-4618-8683-86ec8f938269 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:39.992624 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372971bb-b89e-4618-8683-86ec8f938269 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:39.997842 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c16c00-1cb2-4236-bb0f-c6c9cd287df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:39.998433 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c16c00-1cb2-4236-bb0f-c6c9cd287df5 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:39.998705 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c16c00-1cb2-4236-bb0f-c6c9cd287df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:39.998968 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c16c00-1cb2-4236-bb0f-c6c9cd287df5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:39.999615 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c16c00-1cb2-4236-bb0f-c6c9cd287df5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:40.001548 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c16c00-1cb2-4236-bb0f-c6c9cd287df5 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:40.029273 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1091/2176] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:43:39 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:43:40.033261 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34230bf3-4997-48e4-8f81-88a0cafceb18 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:40.044371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34230bf3-4997-48e4-8f81-88a0cafceb18 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:40.103883 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d384d9-db42-4b45-bce1-d843179e3759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:40.104491 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d384d9-db42-4b45-bce1-d843179e3759 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:40.104794 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d384d9-db42-4b45-bce1-d843179e3759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:40.105071 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d384d9-db42-4b45-bce1-d843179e3759 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:40.105703 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d384d9-db42-4b45-bce1-d843179e3759 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:40.107651 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d384d9-db42-4b45-bce1-d843179e3759 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:40.130334 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1086/2177] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:43:40 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:43:40.134537 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c16c00-1cb2-4236-bb0f-c6c9cd287df5 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:40.145330 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c16c00-1cb2-4236-bb0f-c6c9cd287df5 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:40.150805 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8bf3fc-a597-4fdc-b05c-e74ac8ee1960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:40.151519 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8bf3fc-a597-4fdc-b05c-e74ac8ee1960 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:40.151838 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8bf3fc-a597-4fdc-b05c-e74ac8ee1960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:40.152150 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8bf3fc-a597-4fdc-b05c-e74ac8ee1960 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:40.152870 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8bf3fc-a597-4fdc-b05c-e74ac8ee1960 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:40.154468 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8bf3fc-a597-4fdc-b05c-e74ac8ee1960 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:40.164834 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1092/2178] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:43:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:43:40.169868 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d384d9-db42-4b45-bce1-d843179e3759 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:40.180329 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d384d9-db42-4b45-bce1-d843179e3759 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:40.185687 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4dc86a-1a6b-4602-b0f5-2ca22589d0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:40.186362 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4dc86a-1a6b-4602-b0f5-2ca22589d0c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:40.186643 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4dc86a-1a6b-4602-b0f5-2ca22589d0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:40.186913 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4dc86a-1a6b-4602-b0f5-2ca22589d0c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:40.187607 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4dc86a-1a6b-4602-b0f5-2ca22589d0c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:40.189330 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4dc86a-1a6b-4602-b0f5-2ca22589d0c2 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:40.200681 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1087/2179] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:43:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:43:40.205168 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8bf3fc-a597-4fdc-b05c-e74ac8ee1960 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:40.216035 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8bf3fc-a597-4fdc-b05c-e74ac8ee1960 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:40.221560 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129a8ef-17f7-45ed-86c6-0bd2a2d4ae4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:40.222284 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129a8ef-17f7-45ed-86c6-0bd2a2d4ae4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:40.222641 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129a8ef-17f7-45ed-86c6-0bd2a2d4ae4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:40.222930 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129a8ef-17f7-45ed-86c6-0bd2a2d4ae4a admin admin] PATH_INFO: `/v3/projects/4ee2066bcc7240f9a63d5c11a2b04a18/users/127c4fdd8f2b40ae8d98208d64a7f20d/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:40.258054 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f129a8ef-17f7-45ed-86c6-0bd2a2d4ae4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ee2066bcc7240f9a63d5c11a2b04a18, user_id=127c4fdd8f2b40ae8d98208d64a7f20d, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:40.261444 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f129a8ef-17f7-45ed-86c6-0bd2a2d4ae4a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:40.284637 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1093/2180] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:43:40 2022] PUT /identity/v3/projects/4ee2066bcc7240f9a63d5c11a2b04a18/users/127c4fdd8f2b40ae8d98208d64a7f20d/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:43:40.288920 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9b1ab-1009-4add-8f03-12ea0fd9d705 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:40.289373 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9b1ab-1009-4add-8f03-12ea0fd9d705 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:40.290397 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9b1ab-1009-4add-8f03-12ea0fd9d705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:40.322948 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-c7b9b1ab-1009-4add-8f03-12ea0fd9d705 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:43:40.324418 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c7b9b1ab-1009-4add-8f03-12ea0fd9d705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:40.385262 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1088/2181] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:43:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:43:40.395292 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129a8ef-17f7-45ed-86c6-0bd2a2d4ae4a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:40.397041 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f129a8ef-17f7-45ed-86c6-0bd2a2d4ae4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:40.458960 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129a8ef-17f7-45ed-86c6-0bd2a2d4ae4a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:40.463232 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3191a-8baf-4bf0-91f3-88af133d8f20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:40.463838 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3191a-8baf-4bf0-91f3-88af133d8f20 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:40.464116 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3191a-8baf-4bf0-91f3-88af133d8f20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:40.464385 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3191a-8baf-4bf0-91f3-88af133d8f20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:40.473433 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de3191a-8baf-4bf0-91f3-88af133d8f20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:40.474814 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de3191a-8baf-4bf0-91f3-88af133d8f20 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:40.484870 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1094/2182] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:43:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:43:44.596238 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96d010e-eebc-4755-adaa-75f7f5b7532b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:44.596834 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96d010e-eebc-4755-adaa-75f7f5b7532b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:44.596834 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96d010e-eebc-4755-adaa-75f7f5b7532b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:44.633386 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a96d010e-eebc-4755-adaa-75f7f5b7532b None None] MFA Rules not processed for user `127c4fdd8f2b40ae8d98208d64a7f20d`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:43:44.635728 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a96d010e-eebc-4755-adaa-75f7f5b7532b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:44.672428 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:43:44.706172 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1089/2183] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:43:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:43:44.710637 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3191a-8baf-4bf0-91f3-88af133d8f20 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:44.721291 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3191a-8baf-4bf0-91f3-88af133d8f20 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:44.726038 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613f80d3-85a4-408d-b7b4-75f120b4fba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:44.726588 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f80d3-85a4-408d-b7b4-75f120b4fba2 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:44.726865 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f80d3-85a4-408d-b7b4-75f120b4fba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:44.727137 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f80d3-85a4-408d-b7b4-75f120b4fba2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:44.727789 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613f80d3-85a4-408d-b7b4-75f120b4fba2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:44.729782 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613f80d3-85a4-408d-b7b4-75f120b4fba2 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:44.758279 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1095/2184] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:43:44 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:43:44.762690 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96d010e-eebc-4755-adaa-75f7f5b7532b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:44.774149 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96d010e-eebc-4755-adaa-75f7f5b7532b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:44.837583 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82023d19-ba0a-44fe-ab15-2ad20b3863cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:44.838192 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82023d19-ba0a-44fe-ab15-2ad20b3863cf admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:44.838465 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82023d19-ba0a-44fe-ab15-2ad20b3863cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:44.838727 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82023d19-ba0a-44fe-ab15-2ad20b3863cf admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:44.839361 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82023d19-ba0a-44fe-ab15-2ad20b3863cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:44.841342 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82023d19-ba0a-44fe-ab15-2ad20b3863cf admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:44.863648 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1090/2185] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:43:44 2022] POST /identity/v3/users => generated 295 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:43:44.867620 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613f80d3-85a4-408d-b7b4-75f120b4fba2 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:44.878913 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613f80d3-85a4-408d-b7b4-75f120b4fba2 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:44.883483 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8945d-7b7b-40eb-a644-5892bc58f610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:44.884092 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8945d-7b7b-40eb-a644-5892bc58f610 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:44.884421 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8945d-7b7b-40eb-a644-5892bc58f610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:44.885227 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8945d-7b7b-40eb-a644-5892bc58f610 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:44.885855 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e8945d-7b7b-40eb-a644-5892bc58f610 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:44.887435 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e8945d-7b7b-40eb-a644-5892bc58f610 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:44.898726 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1096/2186] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:43:44.903048 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82023d19-ba0a-44fe-ab15-2ad20b3863cf admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:44.913278 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82023d19-ba0a-44fe-ab15-2ad20b3863cf admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:44.918649 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f25aee3-1b7e-4844-a16f-1c97350ec80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:44.919324 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25aee3-1b7e-4844-a16f-1c97350ec80a admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:44.919602 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25aee3-1b7e-4844-a16f-1c97350ec80a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:44.919871 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25aee3-1b7e-4844-a16f-1c97350ec80a admin admin] PATH_INFO: `/v3/projects/5ffb0db501244b93bc6260ea11738c79/users/df8e713098ea4f0dad0d9914cca6e44b/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:44.936684 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f25aee3-1b7e-4844-a16f-1c97350ec80a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ffb0db501244b93bc6260ea11738c79, user_id=df8e713098ea4f0dad0d9914cca6e44b, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:44.939390 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f25aee3-1b7e-4844-a16f-1c97350ec80a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:44.953604 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1091/2187] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:43:44 2022] PUT /identity/v3/projects/5ffb0db501244b93bc6260ea11738c79/users/df8e713098ea4f0dad0d9914cca6e44b/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:43:44.957518 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8945d-7b7b-40eb-a644-5892bc58f610 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:44.967925 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8945d-7b7b-40eb-a644-5892bc58f610 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:45.025427 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c9cd9e-4516-4fd2-ad18-7d9151f88c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:45.026037 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9cd9e-4516-4fd2-ad18-7d9151f88c71 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:45.026316 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9cd9e-4516-4fd2-ad18-7d9151f88c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:45.026586 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9cd9e-4516-4fd2-ad18-7d9151f88c71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:45.027213 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c9cd9e-4516-4fd2-ad18-7d9151f88c71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:45.028834 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c9cd9e-4516-4fd2-ad18-7d9151f88c71 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:45.034460 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:43:45.040405 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1097/2188] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:43:45.044655 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f25aee3-1b7e-4844-a16f-1c97350ec80a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:45.055200 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f25aee3-1b7e-4844-a16f-1c97350ec80a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:45.060969 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11360ea-a55d-44ac-b789-7145be5c6b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:45.061651 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11360ea-a55d-44ac-b789-7145be5c6b32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:45.061931 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11360ea-a55d-44ac-b789-7145be5c6b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:45.062203 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11360ea-a55d-44ac-b789-7145be5c6b32 admin admin] PATH_INFO: `/v3/projects/5ffb0db501244b93bc6260ea11738c79/users/df8e713098ea4f0dad0d9914cca6e44b/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:45.064192 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11360ea-a55d-44ac-b789-7145be5c6b32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ffb0db501244b93bc6260ea11738c79, user_id=df8e713098ea4f0dad0d9914cca6e44b, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:45.067272 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11360ea-a55d-44ac-b789-7145be5c6b32 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:45.079813 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1092/2189] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:43:45 2022] PUT /identity/v3/projects/5ffb0db501244b93bc6260ea11738c79/users/df8e713098ea4f0dad0d9914cca6e44b/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:43:47.438563 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c9cd9e-4516-4fd2-ad18-7d9151f88c71 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:47.440802 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-19c9cd9e-4516-4fd2-ad18-7d9151f88c71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:47.469226 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c9cd9e-4516-4fd2-ad18-7d9151f88c71 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:47.553969 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87085084-47b6-4ccb-8ff3-c907851b0843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:47.554820 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87085084-47b6-4ccb-8ff3-c907851b0843 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:47.555409 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87085084-47b6-4ccb-8ff3-c907851b0843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:47.555849 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87085084-47b6-4ccb-8ff3-c907851b0843 admin admin] PATH_INFO: `/v3/users/5e2f89388c744b4cb8f291bd04c63d60` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:47.557982 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87085084-47b6-4ccb-8ff3-c907851b0843 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e2f89388c744b4cb8f291bd04c63d60)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:47.560120 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87085084-47b6-4ccb-8ff3-c907851b0843 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:47.631427 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-87085084-47b6-4ccb-8ff3-c907851b0843 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e2f89388c744b4cb8f291bd04c63d60'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:47.639793 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-87085084-47b6-4ccb-8ff3-c907851b0843 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e2f89388c744b4cb8f291bd04c63d60'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:47.654442 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-87085084-47b6-4ccb-8ff3-c907851b0843 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e2f89388c744b4cb8f291bd04c63d60'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:47.665255 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-87085084-47b6-4ccb-8ff3-c907851b0843 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e2f89388c744b4cb8f291bd04c63d60'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:47.667043 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-87085084-47b6-4ccb-8ff3-c907851b0843 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e2f89388c744b4cb8f291bd04c63d60'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:47.676135 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1098/2190] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:43:47 2022] DELETE /identity/v3/users/5e2f89388c744b4cb8f291bd04c63d60 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:43:47.681748 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8bf8d-0f79-464d-b8a1-60674d027b3f None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:47.682565 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8bf8d-0f79-464d-b8a1-60674d027b3f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:47.683502 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8bf8d-0f79-464d-b8a1-60674d027b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:47.731097 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-5fb8bf8d-0f79-464d-b8a1-60674d027b3f None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:43:47.732315 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5fb8bf8d-0f79-464d-b8a1-60674d027b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:47.823096 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1093/2191] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:43:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:43:47.835386 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87085084-47b6-4ccb-8ff3-c907851b0843 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:47.838391 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-87085084-47b6-4ccb-8ff3-c907851b0843 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:47.922377 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87085084-47b6-4ccb-8ff3-c907851b0843 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:47.927992 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d04b4b-5118-4724-ba80-87b3eed9eab7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:47.928770 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d04b4b-5118-4724-ba80-87b3eed9eab7 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:47.929494 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d04b4b-5118-4724-ba80-87b3eed9eab7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:47.930118 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d04b4b-5118-4724-ba80-87b3eed9eab7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:47.945574 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d04b4b-5118-4724-ba80-87b3eed9eab7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:47.947870 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d04b4b-5118-4724-ba80-87b3eed9eab7 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:47.976612 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1099/2192] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:43:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:43:48.555406 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8bf8d-0f79-464d-b8a1-60674d027b3f None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:48.557820 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5fb8bf8d-0f79-464d-b8a1-60674d027b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:48.578809 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8bf8d-0f79-464d-b8a1-60674d027b3f None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:48.592042 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54a12ea-e016-40f6-b0c6-6e014ba05b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:48.593015 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a12ea-e016-40f6-b0c6-6e014ba05b7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:48.593627 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a12ea-e016-40f6-b0c6-6e014ba05b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:48.594379 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a12ea-e016-40f6-b0c6-6e014ba05b7e admin admin] PATH_INFO: `/v3/projects/cf2bcb07297f478aa3dffe15bc931d53` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:48.595898 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54a12ea-e016-40f6-b0c6-6e014ba05b7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf2bcb07297f478aa3dffe15bc931d53)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:48.600850 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54a12ea-e016-40f6-b0c6-6e014ba05b7e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:48.648033 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:43:48.693805 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f54a12ea-e016-40f6-b0c6-6e014ba05b7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf2bcb07297f478aa3dffe15bc931d53', 'request_id': 'req-f54a12ea-e016-40f6-b0c6-6e014ba05b7e'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:48.711855 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f54a12ea-e016-40f6-b0c6-6e014ba05b7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf2bcb07297f478aa3dffe15bc931d53', 'request_id': 'req-f54a12ea-e016-40f6-b0c6-6e014ba05b7e'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:48.720360 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f54a12ea-e016-40f6-b0c6-6e014ba05b7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf2bcb07297f478aa3dffe15bc931d53', 'request_id': 'req-f54a12ea-e016-40f6-b0c6-6e014ba05b7e'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:48.740880 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f54a12ea-e016-40f6-b0c6-6e014ba05b7e admin admin] The token cache is being invalidate because project cf2bcb07297f478aa3dffe15bc931d53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:43:48.741428 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f54a12ea-e016-40f6-b0c6-6e014ba05b7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:48.742511 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f54a12ea-e016-40f6-b0c6-6e014ba05b7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:43:48.744980 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1094/2193] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:43:48 2022] DELETE /identity/v3/projects/cf2bcb07297f478aa3dffe15bc931d53 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:43:48.749859 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42284fb-ae18-4406-a657-69cda1cecee8 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:48.750734 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42284fb-ae18-4406-a657-69cda1cecee8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:48.751159 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42284fb-ae18-4406-a657-69cda1cecee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:48.777658 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-e42284fb-ae18-4406-a657-69cda1cecee8 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:43:48.779039 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e42284fb-ae18-4406-a657-69cda1cecee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:48.800414 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:43:48.860509 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1100/2194] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:43:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:43:48.870346 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7509e-1219-4e89-93cf-648e6e131780 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:48.870580 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7509e-1219-4e89-93cf-648e6e131780 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:48.870776 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7509e-1219-4e89-93cf-648e6e131780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:48.936133 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-3ef7509e-1219-4e89-93cf-648e6e131780 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:43:48.941466 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3ef7509e-1219-4e89-93cf-648e6e131780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:48.958170 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1095/2195] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:43:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:43:48.962986 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42284fb-ae18-4406-a657-69cda1cecee8 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:48.984131 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42284fb-ae18-4406-a657-69cda1cecee8 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:48.997850 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b73b9-2758-43d2-9f0b-203c2d9d0d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:48.998837 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b73b9-2758-43d2-9f0b-203c2d9d0d1a admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:48.999333 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b73b9-2758-43d2-9f0b-203c2d9d0d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:48.999797 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b73b9-2758-43d2-9f0b-203c2d9d0d1a admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:49.000692 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581b73b9-2758-43d2-9f0b-203c2d9d0d1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:43:49.001206 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581b73b9-2758-43d2-9f0b-203c2d9d0d1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:49.002994 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581b73b9-2758-43d2-9f0b-203c2d9d0d1a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:49.034021 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1101/2196] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:43:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:43:49.034520 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef7509e-1219-4e89-93cf-648e6e131780 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:49.054015 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef7509e-1219-4e89-93cf-648e6e131780 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:49.067420 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad873c2-00c0-48a0-934e-1ebf7b4158ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:49.069103 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad873c2-00c0-48a0-934e-1ebf7b4158ac admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:49.070555 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad873c2-00c0-48a0-934e-1ebf7b4158ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:49.070555 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad873c2-00c0-48a0-934e-1ebf7b4158ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:49.070988 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad873c2-00c0-48a0-934e-1ebf7b4158ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:49.078488 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad873c2-00c0-48a0-934e-1ebf7b4158ac admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:49.115720 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1096/2197] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:43:49 2022] POST /identity/v3/projects => generated 381 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:43:49.120434 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b73b9-2758-43d2-9f0b-203c2d9d0d1a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:49.137000 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b73b9-2758-43d2-9f0b-203c2d9d0d1a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:49.232983 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bfa716-c097-473e-9a2c-27b3061f743b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:49.233730 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bfa716-c097-473e-9a2c-27b3061f743b admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:49.235170 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bfa716-c097-473e-9a2c-27b3061f743b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:49.235825 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bfa716-c097-473e-9a2c-27b3061f743b admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:49.237289 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bfa716-c097-473e-9a2c-27b3061f743b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:49.240325 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bfa716-c097-473e-9a2c-27b3061f743b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:49.278320 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1102/2198] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:43:49 2022] POST /identity/v3/users => generated 291 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:43:49.284541 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad873c2-00c0-48a0-934e-1ebf7b4158ac admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:49.295568 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad873c2-00c0-48a0-934e-1ebf7b4158ac admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:49.304818 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa53363-d606-447c-8c6d-b3439a86a80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:49.305471 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa53363-d606-447c-8c6d-b3439a86a80e admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:49.306188 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa53363-d606-447c-8c6d-b3439a86a80e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:49.306188 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa53363-d606-447c-8c6d-b3439a86a80e admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:49.307269 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa53363-d606-447c-8c6d-b3439a86a80e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:49.312252 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa53363-d606-447c-8c6d-b3439a86a80e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:49.339958 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1097/2199] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:43:49 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:43:49.343823 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bfa716-c097-473e-9a2c-27b3061f743b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:49.354815 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bfa716-c097-473e-9a2c-27b3061f743b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:49.373133 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4ccc55-4ace-4e81-a06c-1559cb3f9d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:49.373774 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ccc55-4ace-4e81-a06c-1559cb3f9d08 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:49.374053 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ccc55-4ace-4e81-a06c-1559cb3f9d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:49.374327 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ccc55-4ace-4e81-a06c-1559cb3f9d08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:49.374993 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4ccc55-4ace-4e81-a06c-1559cb3f9d08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:49.381255 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4ccc55-4ace-4e81-a06c-1559cb3f9d08 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:49.403519 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1103/2200] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:43:49 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:43:49.406889 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa53363-d606-447c-8c6d-b3439a86a80e admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:49.424966 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa53363-d606-447c-8c6d-b3439a86a80e admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:49.430569 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cd51a5-1bda-47c3-9ce5-e50d14bfe457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:49.432134 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd51a5-1bda-47c3-9ce5-e50d14bfe457 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:49.432416 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd51a5-1bda-47c3-9ce5-e50d14bfe457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:49.432728 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd51a5-1bda-47c3-9ce5-e50d14bfe457 admin admin] PATH_INFO: `/v3/projects/f5ba90d85a624d1d9c126d71ac2342b1/users/07271b6c3e614125a157e3c76fa0fe15/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:49.455302 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cd51a5-1bda-47c3-9ce5-e50d14bfe457 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5ba90d85a624d1d9c126d71ac2342b1, user_id=07271b6c3e614125a157e3c76fa0fe15, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:49.458575 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cd51a5-1bda-47c3-9ce5-e50d14bfe457 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:49.496086 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86169db-7cb8-4e12-a27d-9aa46c16437a None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:49.496406 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86169db-7cb8-4e12-a27d-9aa46c16437a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:49.496678 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86169db-7cb8-4e12-a27d-9aa46c16437a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:49.499403 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1098/2201] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:43:49 2022] PUT /identity/v3/projects/f5ba90d85a624d1d9c126d71ac2342b1/users/07271b6c3e614125a157e3c76fa0fe15/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:43:49.551493 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-b86169db-7cb8-4e12-a27d-9aa46c16437a None None] MFA Rules not processed for user `07271b6c3e614125a157e3c76fa0fe15`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:43:49.552882 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b86169db-7cb8-4e12-a27d-9aa46c16437a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:49.662169 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1104/2202] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:43:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:43:49.663858 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cd51a5-1bda-47c3-9ce5-e50d14bfe457 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:49.754774 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cd51a5-1bda-47c3-9ce5-e50d14bfe457 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:49.770736 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547cfe79-11a0-4f10-868b-5628e9819c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:49.771512 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547cfe79-11a0-4f10-868b-5628e9819c1a admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:49.771894 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547cfe79-11a0-4f10-868b-5628e9819c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:49.772285 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547cfe79-11a0-4f10-868b-5628e9819c1a admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:49.773076 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547cfe79-11a0-4f10-868b-5628e9819c1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:49.777505 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547cfe79-11a0-4f10-868b-5628e9819c1a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:49.817092 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1099/2203] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:43:49 2022] POST /identity/v3/projects => generated 379 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:43:49.821310 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86169db-7cb8-4e12-a27d-9aa46c16437a None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:49.835841 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86169db-7cb8-4e12-a27d-9aa46c16437a None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:49.866212 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:43:49.935726 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c6220-7c05-4a52-9835-9bcfa47a7e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:49.936457 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c6220-7c05-4a52-9835-9bcfa47a7e61 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:49.936867 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c6220-7c05-4a52-9835-9bcfa47a7e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:49.937314 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c6220-7c05-4a52-9835-9bcfa47a7e61 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:49.938236 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2c6220-7c05-4a52-9835-9bcfa47a7e61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:49.940455 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2c6220-7c05-4a52-9835-9bcfa47a7e61 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:49.992754 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1105/2204] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:43:49 2022] POST /identity/v3/users => generated 296 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:43:49.997239 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547cfe79-11a0-4f10-868b-5628e9819c1a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:50.026047 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547cfe79-11a0-4f10-868b-5628e9819c1a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:50.035056 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb1b2f-f8de-47bb-8916-e0529e69fb4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:50.037014 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb1b2f-f8de-47bb-8916-e0529e69fb4c admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:50.037521 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb1b2f-f8de-47bb-8916-e0529e69fb4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:50.038971 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb1b2f-f8de-47bb-8916-e0529e69fb4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:50.039948 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbb1b2f-f8de-47bb-8916-e0529e69fb4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:50.045703 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbb1b2f-f8de-47bb-8916-e0529e69fb4c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:50.065163 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b57331-3767-4a84-98fa-3c05de3e5cff None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:50.066193 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b57331-3767-4a84-98fa-3c05de3e5cff None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:50.066193 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b57331-3767-4a84-98fa-3c05de3e5cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:50.074391 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1100/2205] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:43:49 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:43:50.078322 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb1b2f-f8de-47bb-8916-e0529e69fb4c admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:50.093407 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb1b2f-f8de-47bb-8916-e0529e69fb4c admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:50.104660 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-72b57331-3767-4a84-98fa-3c05de3e5cff None None] MFA Rules not processed for user `df8e713098ea4f0dad0d9914cca6e44b`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:43:50.105558 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b49fb61-c175-4bad-af70-7614b973182d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:50.106271 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-72b57331-3767-4a84-98fa-3c05de3e5cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:50.106398 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49fb61-c175-4bad-af70-7614b973182d admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:50.106587 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49fb61-c175-4bad-af70-7614b973182d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:50.107942 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49fb61-c175-4bad-af70-7614b973182d admin admin] PATH_INFO: `/v3/projects/64f82c125b8e42d0a0832a174d16ca1d/users/41237196d6d749f98002875075b5ddc5/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:50.138073 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b49fb61-c175-4bad-af70-7614b973182d admin admin] RBAC: Authorizing `identity:create_grant(project_id=64f82c125b8e42d0a0832a174d16ca1d, user_id=41237196d6d749f98002875075b5ddc5, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:50.141151 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b49fb61-c175-4bad-af70-7614b973182d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:50.156814 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:43:50.165730 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1101/2206] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:43:50 2022] PUT /identity/v3/projects/64f82c125b8e42d0a0832a174d16ca1d/users/41237196d6d749f98002875075b5ddc5/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:43:50.170371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b49fb61-c175-4bad-af70-7614b973182d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:50.232284 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1106/2207] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:43:50 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:43:50.249708 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b57331-3767-4a84-98fa-3c05de3e5cff None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:50.251379 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-72b57331-3767-4a84-98fa-3c05de3e5cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:50.288281 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b49fb61-c175-4bad-af70-7614b973182d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:50.294662 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff27782-ecd9-41f2-a4f3-4093157c93be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:50.295278 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff27782-ecd9-41f2-a4f3-4093157c93be admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:50.295560 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff27782-ecd9-41f2-a4f3-4093157c93be admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:50.295835 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff27782-ecd9-41f2-a4f3-4093157c93be admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:50.296480 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff27782-ecd9-41f2-a4f3-4093157c93be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:50.298564 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff27782-ecd9-41f2-a4f3-4093157c93be admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:50.313196 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1102/2208] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:43:50 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:43:50.317252 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff27782-ecd9-41f2-a4f3-4093157c93be admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:50.330313 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff27782-ecd9-41f2-a4f3-4093157c93be admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:50.336329 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bb878a-c0e5-4a8c-b371-4c327818f3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:50.337039 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb878a-c0e5-4a8c-b371-4c327818f3a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:50.337346 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb878a-c0e5-4a8c-b371-4c327818f3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:50.337672 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb878a-c0e5-4a8c-b371-4c327818f3a1 admin admin] PATH_INFO: `/v3/projects/64f82c125b8e42d0a0832a174d16ca1d/users/41237196d6d749f98002875075b5ddc5/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:50.339664 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bb878a-c0e5-4a8c-b371-4c327818f3a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64f82c125b8e42d0a0832a174d16ca1d, user_id=41237196d6d749f98002875075b5ddc5, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:50.342341 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bb878a-c0e5-4a8c-b371-4c327818f3a1 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:50.353633 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1103/2209] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:43:50 2022] PUT /identity/v3/projects/64f82c125b8e42d0a0832a174d16ca1d/users/41237196d6d749f98002875075b5ddc5/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:43:50.363692 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13491105-a871-4eae-87e1-bc5641ac6969 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:50.363977 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13491105-a871-4eae-87e1-bc5641ac6969 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:50.364556 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13491105-a871-4eae-87e1-bc5641ac6969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:50.370457 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b57331-3767-4a84-98fa-3c05de3e5cff None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:50.408303 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-13491105-a871-4eae-87e1-bc5641ac6969 None None] MFA Rules not processed for user `41237196d6d749f98002875075b5ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:43:50.411668 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-13491105-a871-4eae-87e1-bc5641ac6969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:50.412830 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:43:50.457676 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c524ee4b-de82-45a9-8a5e-3f2d06130df3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe4a6e4b6184826b30ac4fc13ad9b1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:50.458301 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c524ee4b-de82-45a9-8a5e-3f2d06130df3 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:50.458587 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c524ee4b-de82-45a9-8a5e-3f2d06130df3 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:50.458861 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c524ee4b-de82-45a9-8a5e-3f2d06130df3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:50.460843 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c524ee4b-de82-45a9-8a5e-3f2d06130df3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:50.537021 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1104/2210] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:43:50 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:43:50.545023 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c524ee4b-de82-45a9-8a5e-3f2d06130df3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:50.546887 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c524ee4b-de82-45a9-8a5e-3f2d06130df3 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:50.550570 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13491105-a871-4eae-87e1-bc5641ac6969 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:50.552986 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-13491105-a871-4eae-87e1-bc5641ac6969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:50.558605 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1107/2211] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:43:50 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:43:50.635192 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13491105-a871-4eae-87e1-bc5641ac6969 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:50.639197 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a3206-660a-4596-b539-025cc7043eb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:50.639887 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a3206-660a-4596-b539-025cc7043eb1 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:50.640239 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a3206-660a-4596-b539-025cc7043eb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:50.640830 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a3206-660a-4596-b539-025cc7043eb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:50.653816 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800a3206-660a-4596-b539-025cc7043eb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:50.655800 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800a3206-660a-4596-b539-025cc7043eb1 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:50.668039 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1105/2212] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:43:50 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 09:43:50.774412 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c524ee4b-de82-45a9-8a5e-3f2d06130df3 service nova] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:50.783745 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c524ee4b-de82-45a9-8a5e-3f2d06130df3 service nova] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:50.788188 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da705846-37b4-4265-b980-eab258f92bc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:50.788963 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da705846-37b4-4265-b980-eab258f92bc8 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:50.789455 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da705846-37b4-4265-b980-eab258f92bc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:50.789588 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da705846-37b4-4265-b980-eab258f92bc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:50.864378 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da705846-37b4-4265-b980-eab258f92bc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:50.865865 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da705846-37b4-4265-b980-eab258f92bc8 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:50.875611 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1108/2213] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:43:50 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:43:54.018813 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a3206-660a-4596-b539-025cc7043eb1 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:43:54.020364 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-800a3206-660a-4596-b539-025cc7043eb1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:54.032120 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:43:54.082157 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a3206-660a-4596-b539-025cc7043eb1 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:43:54.086102 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf6b22-01b8-4b16-9856-0755a7c00fc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:43:54.086714 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf6b22-01b8-4b16-9856-0755a7c00fc0 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:43:54.087526 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf6b22-01b8-4b16-9856-0755a7c00fc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:43:54.087802 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf6b22-01b8-4b16-9856-0755a7c00fc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:43:54.089589 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3cbf6b22-01b8-4b16-9856-0755a7c00fc0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:43:54.152084 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbf6b22-01b8-4b16-9856-0755a7c00fc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:43:54.153745 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbf6b22-01b8-4b16-9856-0755a7c00fc0 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:43:54.180282 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1106/2214] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:43:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:44:00.874046 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=70 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:00.965049 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da705846-37b4-4265-b980-eab258f92bc8 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:01.026993 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da705846-37b4-4265-b980-eab258f92bc8 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:01.031629 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb77d7e-158f-41ac-9b02-31b26c43b000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:01.032247 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb77d7e-158f-41ac-9b02-31b26c43b000 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:01.032541 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb77d7e-158f-41ac-9b02-31b26c43b000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:01.032831 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb77d7e-158f-41ac-9b02-31b26c43b000 admin admin] PATH_INFO: `/v3/users/07271b6c3e614125a157e3c76fa0fe15` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:01.033938 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb77d7e-158f-41ac-9b02-31b26c43b000 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07271b6c3e614125a157e3c76fa0fe15)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:01.036060 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb77d7e-158f-41ac-9b02-31b26c43b000 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:01.083561 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0eb77d7e-158f-41ac-9b02-31b26c43b000 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07271b6c3e614125a157e3c76fa0fe15'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:01.090464 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0eb77d7e-158f-41ac-9b02-31b26c43b000 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07271b6c3e614125a157e3c76fa0fe15'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:01.102212 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0eb77d7e-158f-41ac-9b02-31b26c43b000 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07271b6c3e614125a157e3c76fa0fe15'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:01.109961 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0eb77d7e-158f-41ac-9b02-31b26c43b000 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07271b6c3e614125a157e3c76fa0fe15'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:01.110663 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0eb77d7e-158f-41ac-9b02-31b26c43b000 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07271b6c3e614125a157e3c76fa0fe15'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:01.116949 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1109/2215] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:44:00 2022] DELETE /identity/v3/users/07271b6c3e614125a157e3c76fa0fe15 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:44:01.122468 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf0a24-a2d0-4800-9789-a4118fa736c0 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:01.122917 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf0a24-a2d0-4800-9789-a4118fa736c0 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:01.123258 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf0a24-a2d0-4800-9789-a4118fa736c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:01.146825 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-4dcf0a24-a2d0-4800-9789-a4118fa736c0 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:01.148059 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4dcf0a24-a2d0-4800-9789-a4118fa736c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:01.217118 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1107/2216] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:44:01.226810 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb77d7e-158f-41ac-9b02-31b26c43b000 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:01.228933 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0eb77d7e-158f-41ac-9b02-31b26c43b000 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:01.291812 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb77d7e-158f-41ac-9b02-31b26c43b000 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:01.295533 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d3b16-b6b6-4345-87fe-fbd04b3b48d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:01.296144 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d3b16-b6b6-4345-87fe-fbd04b3b48d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:01.296418 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d3b16-b6b6-4345-87fe-fbd04b3b48d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:01.296726 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d3b16-b6b6-4345-87fe-fbd04b3b48d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:01.305658 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d3b16-b6b6-4345-87fe-fbd04b3b48d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:01.307659 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d3b16-b6b6-4345-87fe-fbd04b3b48d3 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:01.320615 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1110/2217] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:44:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:44:01.373499 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf0a24-a2d0-4800-9789-a4118fa736c0 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:01.374949 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4dcf0a24-a2d0-4800-9789-a4118fa736c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:01.404928 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf0a24-a2d0-4800-9789-a4118fa736c0 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:01.410814 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba03ddc-6ab0-43df-8240-3ff12ed1c228 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:01.412314 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba03ddc-6ab0-43df-8240-3ff12ed1c228 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:01.412639 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba03ddc-6ab0-43df-8240-3ff12ed1c228 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:01.412916 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba03ddc-6ab0-43df-8240-3ff12ed1c228 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:01.414774 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-aba03ddc-6ab0-43df-8240-3ff12ed1c228 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:01.428093 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba03ddc-6ab0-43df-8240-3ff12ed1c228 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:01.429588 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba03ddc-6ab0-43df-8240-3ff12ed1c228 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:01.440562 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1108/2218] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:44:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:44:01.611482 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d3b16-b6b6-4345-87fe-fbd04b3b48d3 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:01.614449 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f97d3b16-b6b6-4345-87fe-fbd04b3b48d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:01.631919 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d3b16-b6b6-4345-87fe-fbd04b3b48d3 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:01.638588 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f177a-1ca0-46b8-822f-34345d12ec6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:01.640403 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f177a-1ca0-46b8-822f-34345d12ec6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:01.640560 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f177a-1ca0-46b8-822f-34345d12ec6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:01.641023 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f177a-1ca0-46b8-822f-34345d12ec6a admin admin] PATH_INFO: `/v3/projects/f5ba90d85a624d1d9c126d71ac2342b1` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:01.643557 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f177a-1ca0-46b8-822f-34345d12ec6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5ba90d85a624d1d9c126d71ac2342b1)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:01.647118 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f177a-1ca0-46b8-822f-34345d12ec6a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:01.676550 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:01.716726 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-415f177a-1ca0-46b8-822f-34345d12ec6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5ba90d85a624d1d9c126d71ac2342b1', 'request_id': 'req-415f177a-1ca0-46b8-822f-34345d12ec6a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:01.722949 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-415f177a-1ca0-46b8-822f-34345d12ec6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5ba90d85a624d1d9c126d71ac2342b1', 'request_id': 'req-415f177a-1ca0-46b8-822f-34345d12ec6a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:01.730301 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-415f177a-1ca0-46b8-822f-34345d12ec6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5ba90d85a624d1d9c126d71ac2342b1', 'request_id': 'req-415f177a-1ca0-46b8-822f-34345d12ec6a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:01.742492 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-415f177a-1ca0-46b8-822f-34345d12ec6a admin admin] The token cache is being invalidate because project f5ba90d85a624d1d9c126d71ac2342b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:44:01.742783 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-415f177a-1ca0-46b8-822f-34345d12ec6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:01.743462 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-415f177a-1ca0-46b8-822f-34345d12ec6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:01.745222 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1111/2219] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:44:01 2022] DELETE /identity/v3/projects/f5ba90d85a624d1d9c126d71ac2342b1 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:44:01.749835 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba03ddc-6ab0-43df-8240-3ff12ed1c228 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:01.760837 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba03ddc-6ab0-43df-8240-3ff12ed1c228 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:01.807630 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:01.819721 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa847b-2ac7-42ec-b043-437c047cea92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:01.820459 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa847b-2ac7-42ec-b043-437c047cea92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:01.820894 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa847b-2ac7-42ec-b043-437c047cea92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:01.821166 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa847b-2ac7-42ec-b043-437c047cea92 admin admin] PATH_INFO: `/v3/users/41237196d6d749f98002875075b5ddc5` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:01.823325 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aa847b-2ac7-42ec-b043-437c047cea92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41237196d6d749f98002875075b5ddc5)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:01.825282 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aa847b-2ac7-42ec-b043-437c047cea92 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:01.872380 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-49aa847b-2ac7-42ec-b043-437c047cea92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41237196d6d749f98002875075b5ddc5'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:01.879199 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-49aa847b-2ac7-42ec-b043-437c047cea92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41237196d6d749f98002875075b5ddc5'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:01.891583 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-49aa847b-2ac7-42ec-b043-437c047cea92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41237196d6d749f98002875075b5ddc5'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:01.901760 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-49aa847b-2ac7-42ec-b043-437c047cea92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41237196d6d749f98002875075b5ddc5'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:01.902483 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-49aa847b-2ac7-42ec-b043-437c047cea92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41237196d6d749f98002875075b5ddc5'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:01.908530 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1109/2220] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:44:01 2022] DELETE /identity/v3/users/41237196d6d749f98002875075b5ddc5 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:44:02.578984 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f177a-1ca0-46b8-822f-34345d12ec6a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:02.580577 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-415f177a-1ca0-46b8-822f-34345d12ec6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:02.594783 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f177a-1ca0-46b8-822f-34345d12ec6a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:02.660455 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38483df8-f22d-4182-b612-210dfa782950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:02.661045 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38483df8-f22d-4182-b612-210dfa782950 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:02.661326 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38483df8-f22d-4182-b612-210dfa782950 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:02.661603 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38483df8-f22d-4182-b612-210dfa782950 admin admin] PATH_INFO: `/v3/projects/64f82c125b8e42d0a0832a174d16ca1d` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:02.662715 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38483df8-f22d-4182-b612-210dfa782950 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64f82c125b8e42d0a0832a174d16ca1d)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:02.664700 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38483df8-f22d-4182-b612-210dfa782950 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:02.757908 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-38483df8-f22d-4182-b612-210dfa782950 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64f82c125b8e42d0a0832a174d16ca1d', 'request_id': 'req-38483df8-f22d-4182-b612-210dfa782950'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:02.777734 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-38483df8-f22d-4182-b612-210dfa782950 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64f82c125b8e42d0a0832a174d16ca1d', 'request_id': 'req-38483df8-f22d-4182-b612-210dfa782950'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:02.786512 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-38483df8-f22d-4182-b612-210dfa782950 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64f82c125b8e42d0a0832a174d16ca1d', 'request_id': 'req-38483df8-f22d-4182-b612-210dfa782950'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:02.799543 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-38483df8-f22d-4182-b612-210dfa782950 admin admin] The token cache is being invalidate because project 64f82c125b8e42d0a0832a174d16ca1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:44:02.799881 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-38483df8-f22d-4182-b612-210dfa782950 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:02.800509 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-38483df8-f22d-4182-b612-210dfa782950 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:02.802369 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1112/2221] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:44:02 2022] DELETE /identity/v3/projects/64f82c125b8e42d0a0832a174d16ca1d => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:44:02.807664 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb6cf9e-0a4d-471b-b3a3-a35b72329d9d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:02.808165 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb6cf9e-0a4d-471b-b3a3-a35b72329d9d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:02.808301 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb6cf9e-0a4d-471b-b3a3-a35b72329d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:02.832700 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-4fb6cf9e-0a4d-471b-b3a3-a35b72329d9d None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:02.834054 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4fb6cf9e-0a4d-471b-b3a3-a35b72329d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:02.904337 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1110/2222] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:44:02.912094 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933dcb9b-99a6-40fc-a581-e31aebe1ae25 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:02.912894 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933dcb9b-99a6-40fc-a581-e31aebe1ae25 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:02.912894 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933dcb9b-99a6-40fc-a581-e31aebe1ae25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:02.933099 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-933dcb9b-99a6-40fc-a581-e31aebe1ae25 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:02.934415 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-933dcb9b-99a6-40fc-a581-e31aebe1ae25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:02.945961 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1113/2223] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:44:02.948340 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb6cf9e-0a4d-471b-b3a3-a35b72329d9d None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:02.958118 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb6cf9e-0a4d-471b-b3a3-a35b72329d9d None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:02.963581 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1555ed-cbda-4165-94f4-5006d3e044c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:02.964189 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1555ed-cbda-4165-94f4-5006d3e044c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:02.964612 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1555ed-cbda-4165-94f4-5006d3e044c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:02.964785 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1555ed-cbda-4165-94f4-5006d3e044c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:02.965435 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1555ed-cbda-4165-94f4-5006d3e044c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:44:02.965736 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1555ed-cbda-4165-94f4-5006d3e044c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:02.967335 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1555ed-cbda-4165-94f4-5006d3e044c2 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:02.986612 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933dcb9b-99a6-40fc-a581-e31aebe1ae25 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:02.997035 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933dcb9b-99a6-40fc-a581-e31aebe1ae25 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:02.997613 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1111/2224] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:44:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:44:03.003576 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b57bea-8714-4bb8-b3ad-c6c74e7ba3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:03.004770 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b57bea-8714-4bb8-b3ad-c6c74e7ba3e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:03.004770 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b57bea-8714-4bb8-b3ad-c6c74e7ba3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:03.005013 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b57bea-8714-4bb8-b3ad-c6c74e7ba3e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:03.005742 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b57bea-8714-4bb8-b3ad-c6c74e7ba3e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:03.008765 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b57bea-8714-4bb8-b3ad-c6c74e7ba3e1 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:03.043774 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1114/2225] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:44:02 2022] POST /identity/v3/projects => generated 363 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:44:03.047998 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1555ed-cbda-4165-94f4-5006d3e044c2 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:03.060556 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1555ed-cbda-4165-94f4-5006d3e044c2 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:03.119503 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:03.126444 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8b1958-2511-40bb-a890-13c4a2cfab66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:03.127054 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8b1958-2511-40bb-a890-13c4a2cfab66 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:03.127332 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8b1958-2511-40bb-a890-13c4a2cfab66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:03.127653 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8b1958-2511-40bb-a890-13c4a2cfab66 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:03.128316 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8b1958-2511-40bb-a890-13c4a2cfab66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:03.130377 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8b1958-2511-40bb-a890-13c4a2cfab66 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:03.155614 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1112/2226] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:44:03 2022] POST /identity/v3/users => generated 282 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:44:03.159811 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b57bea-8714-4bb8-b3ad-c6c74e7ba3e1 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:03.170667 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b57bea-8714-4bb8-b3ad-c6c74e7ba3e1 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:03.176440 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b199e4-9986-4e0e-8de5-2b1ce7511391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:03.177184 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b199e4-9986-4e0e-8de5-2b1ce7511391 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:03.177561 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b199e4-9986-4e0e-8de5-2b1ce7511391 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:03.177928 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b199e4-9986-4e0e-8de5-2b1ce7511391 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:03.178664 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b199e4-9986-4e0e-8de5-2b1ce7511391 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:03.180371 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b199e4-9986-4e0e-8de5-2b1ce7511391 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:03.182484 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:03.198504 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1115/2227] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:44:03 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:44:03.204410 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8b1958-2511-40bb-a890-13c4a2cfab66 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:03.220922 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8b1958-2511-40bb-a890-13c4a2cfab66 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:03.223554 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4c92cc-b09f-4666-95a4-b89c886a3b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:03.224349 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c92cc-b09f-4666-95a4-b89c886a3b21 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:03.224655 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c92cc-b09f-4666-95a4-b89c886a3b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:03.224928 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c92cc-b09f-4666-95a4-b89c886a3b21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:03.226185 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4c92cc-b09f-4666-95a4-b89c886a3b21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:03.228368 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4c92cc-b09f-4666-95a4-b89c886a3b21 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:03.240557 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1113/2228] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:44:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:44:03.244042 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b199e4-9986-4e0e-8de5-2b1ce7511391 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:03.255279 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b199e4-9986-4e0e-8de5-2b1ce7511391 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:03.263287 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cbf587-b910-4a37-bb16-dac09f8a26fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:03.264213 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cbf587-b910-4a37-bb16-dac09f8a26fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:03.264905 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cbf587-b910-4a37-bb16-dac09f8a26fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:03.265336 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cbf587-b910-4a37-bb16-dac09f8a26fb admin admin] PATH_INFO: `/v3/projects/43bcfda94ed74808bb64e062f0fdc93f/users/55680bdade0d43658bd080ccd2f362b0/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:03.299049 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cbf587-b910-4a37-bb16-dac09f8a26fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=43bcfda94ed74808bb64e062f0fdc93f, user_id=55680bdade0d43658bd080ccd2f362b0, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:03.301790 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cbf587-b910-4a37-bb16-dac09f8a26fb admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:03.318344 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1116/2229] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:44:03 2022] PUT /identity/v3/projects/43bcfda94ed74808bb64e062f0fdc93f/users/55680bdade0d43658bd080ccd2f362b0/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:44:03.330295 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13328356-73cd-47e7-9fae-cdd97268cf0d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:03.330616 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13328356-73cd-47e7-9fae-cdd97268cf0d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:03.331462 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13328356-73cd-47e7-9fae-cdd97268cf0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:03.364720 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-13328356-73cd-47e7-9fae-cdd97268cf0d None None] MFA Rules not processed for user `55680bdade0d43658bd080ccd2f362b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:03.366024 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-13328356-73cd-47e7-9fae-cdd97268cf0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:03.445545 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1114/2230] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:44:03.450408 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cbf587-b910-4a37-bb16-dac09f8a26fb admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:03.547015 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cbf587-b910-4a37-bb16-dac09f8a26fb admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:03.552709 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0f62ec-1db2-4aa3-8f2e-2ad2cfdc2b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:03.554569 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f62ec-1db2-4aa3-8f2e-2ad2cfdc2b6a admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:03.555080 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f62ec-1db2-4aa3-8f2e-2ad2cfdc2b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:03.555816 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f62ec-1db2-4aa3-8f2e-2ad2cfdc2b6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:03.557421 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0f62ec-1db2-4aa3-8f2e-2ad2cfdc2b6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:03.559751 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0f62ec-1db2-4aa3-8f2e-2ad2cfdc2b6a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:03.596737 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1117/2231] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:44:03 2022] POST /identity/v3/projects => generated 365 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:44:03.598416 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13328356-73cd-47e7-9fae-cdd97268cf0d None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:03.610893 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13328356-73cd-47e7-9fae-cdd97268cf0d None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:03.686580 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a144de95-8bbc-4d6a-b1d6-5892fb880ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:03.687498 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a144de95-8bbc-4d6a-b1d6-5892fb880ee4 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:03.687880 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a144de95-8bbc-4d6a-b1d6-5892fb880ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:03.688253 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a144de95-8bbc-4d6a-b1d6-5892fb880ee4 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:03.689156 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a144de95-8bbc-4d6a-b1d6-5892fb880ee4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:03.691544 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a144de95-8bbc-4d6a-b1d6-5892fb880ee4 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:03.724745 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0f62ec-1db2-4aa3-8f2e-2ad2cfdc2b6a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:03.726607 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1115/2232] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:44:03 2022] POST /identity/v3/users => generated 289 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:44:03.736858 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0f62ec-1db2-4aa3-8f2e-2ad2cfdc2b6a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:03.742435 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994076d9-f205-4533-a47a-5ebb9fccfdac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:03.743206 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994076d9-f205-4533-a47a-5ebb9fccfdac admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:03.743732 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994076d9-f205-4533-a47a-5ebb9fccfdac admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:03.744054 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994076d9-f205-4533-a47a-5ebb9fccfdac admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:03.744887 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994076d9-f205-4533-a47a-5ebb9fccfdac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:03.747500 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994076d9-f205-4533-a47a-5ebb9fccfdac admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:03.773770 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1118/2233] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:44:03 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:44:03.774305 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a144de95-8bbc-4d6a-b1d6-5892fb880ee4 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:03.802968 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a144de95-8bbc-4d6a-b1d6-5892fb880ee4 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:03.811884 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fd5f0f-d982-4dca-9b29-7bc25586048b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:03.812583 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd5f0f-d982-4dca-9b29-7bc25586048b admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:03.812873 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd5f0f-d982-4dca-9b29-7bc25586048b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:03.813146 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd5f0f-d982-4dca-9b29-7bc25586048b admin admin] PATH_INFO: `/v3/projects/efc4f25284d54c0fb42ecdb1ae3199a6/users/6550536c78f74d4cb1e7d7247663b644/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:03.833684 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fd5f0f-d982-4dca-9b29-7bc25586048b admin admin] RBAC: Authorizing `identity:create_grant(project_id=efc4f25284d54c0fb42ecdb1ae3199a6, user_id=6550536c78f74d4cb1e7d7247663b644, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:03.836817 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fd5f0f-d982-4dca-9b29-7bc25586048b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:03.867314 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1116/2234] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:44:03 2022] PUT /identity/v3/projects/efc4f25284d54c0fb42ecdb1ae3199a6/users/6550536c78f74d4cb1e7d7247663b644/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:44:03.872433 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994076d9-f205-4533-a47a-5ebb9fccfdac admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:03.884594 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994076d9-f205-4533-a47a-5ebb9fccfdac admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:03.961050 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fade5ba-938a-48ef-9277-9ea9963ac501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:03.962889 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fade5ba-938a-48ef-9277-9ea9963ac501 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:03.963443 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fade5ba-938a-48ef-9277-9ea9963ac501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:03.966494 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fade5ba-938a-48ef-9277-9ea9963ac501 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:03.967213 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fade5ba-938a-48ef-9277-9ea9963ac501 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:03.969503 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fade5ba-938a-48ef-9277-9ea9963ac501 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:03.986120 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1119/2235] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:44:03 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:44:03.987272 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fd5f0f-d982-4dca-9b29-7bc25586048b admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:04.005474 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fd5f0f-d982-4dca-9b29-7bc25586048b admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:04.011949 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acb4976-8417-40a8-8b65-77d6b462c9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:04.012713 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb4976-8417-40a8-8b65-77d6b462c9cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:04.012996 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb4976-8417-40a8-8b65-77d6b462c9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:04.013345 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb4976-8417-40a8-8b65-77d6b462c9cf admin admin] PATH_INFO: `/v3/projects/efc4f25284d54c0fb42ecdb1ae3199a6/users/6550536c78f74d4cb1e7d7247663b644/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:04.015487 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acb4976-8417-40a8-8b65-77d6b462c9cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=efc4f25284d54c0fb42ecdb1ae3199a6, user_id=6550536c78f74d4cb1e7d7247663b644, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:04.018162 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acb4976-8417-40a8-8b65-77d6b462c9cf admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:04.030078 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1117/2236] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:44:03 2022] PUT /identity/v3/projects/efc4f25284d54c0fb42ecdb1ae3199a6/users/6550536c78f74d4cb1e7d7247663b644/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:44:04.040223 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766f2749-0fe9-4504-b675-a1218849391c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:04.040557 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766f2749-0fe9-4504-b675-a1218849391c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:04.040834 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766f2749-0fe9-4504-b675-a1218849391c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:04.075636 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-766f2749-0fe9-4504-b675-a1218849391c None None] MFA Rules not processed for user `6550536c78f74d4cb1e7d7247663b644`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:04.077035 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-766f2749-0fe9-4504-b675-a1218849391c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:04.107802 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:04.152049 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1120/2237] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:04 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:44:04.163430 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acb4976-8417-40a8-8b65-77d6b462c9cf admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:04.165458 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6acb4976-8417-40a8-8b65-77d6b462c9cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:04.186126 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:04.264542 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acb4976-8417-40a8-8b65-77d6b462c9cf admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:04.273943 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aeb670-efa2-4531-a59b-1dc070b1bcee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:04.275376 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aeb670-efa2-4531-a59b-1dc070b1bcee service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:04.275736 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aeb670-efa2-4531-a59b-1dc070b1bcee service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:04.276077 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aeb670-efa2-4531-a59b-1dc070b1bcee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:04.287784 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aeb670-efa2-4531-a59b-1dc070b1bcee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:04.289581 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aeb670-efa2-4531-a59b-1dc070b1bcee service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:04.308749 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1118/2238] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:44:04 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:44:04.825021 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766f2749-0fe9-4504-b675-a1218849391c None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:04.928700 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766f2749-0fe9-4504-b675-a1218849391c None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:04.934264 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dd0ed5-7894-4f2e-809c-2b583322bd1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:04.934907 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dd0ed5-7894-4f2e-809c-2b583322bd1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:04.935182 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dd0ed5-7894-4f2e-809c-2b583322bd1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:04.935460 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dd0ed5-7894-4f2e-809c-2b583322bd1c admin admin] PATH_INFO: `/v3/users/55680bdade0d43658bd080ccd2f362b0` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:04.936642 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dd0ed5-7894-4f2e-809c-2b583322bd1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=55680bdade0d43658bd080ccd2f362b0)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:04.950767 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dd0ed5-7894-4f2e-809c-2b583322bd1c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:05.019899 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d5dd0ed5-7894-4f2e-809c-2b583322bd1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55680bdade0d43658bd080ccd2f362b0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:05.033866 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d5dd0ed5-7894-4f2e-809c-2b583322bd1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55680bdade0d43658bd080ccd2f362b0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:05.049127 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d5dd0ed5-7894-4f2e-809c-2b583322bd1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55680bdade0d43658bd080ccd2f362b0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:05.066519 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d5dd0ed5-7894-4f2e-809c-2b583322bd1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55680bdade0d43658bd080ccd2f362b0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:05.068609 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d5dd0ed5-7894-4f2e-809c-2b583322bd1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55680bdade0d43658bd080ccd2f362b0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:05.079080 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1121/2239] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:44:04 2022] DELETE /identity/v3/users/55680bdade0d43658bd080ccd2f362b0 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:44:05.085960 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a21b97-b1c0-4dec-a2db-4f77c720baab None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:05.086756 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a21b97-b1c0-4dec-a2db-4f77c720baab None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:05.087032 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a21b97-b1c0-4dec-a2db-4f77c720baab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:05.111486 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-60a21b97-b1c0-4dec-a2db-4f77c720baab None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:05.113757 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-60a21b97-b1c0-4dec-a2db-4f77c720baab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:05.183166 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1119/2240] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:44:05.193406 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dd0ed5-7894-4f2e-809c-2b583322bd1c admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:05.194948 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d5dd0ed5-7894-4f2e-809c-2b583322bd1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:05.250196 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:05.257972 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dd0ed5-7894-4f2e-809c-2b583322bd1c admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:05.265483 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b26caf-3add-4cdb-bb87-df8449e9ec54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:05.266182 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b26caf-3add-4cdb-bb87-df8449e9ec54 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:05.266536 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b26caf-3add-4cdb-bb87-df8449e9ec54 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:05.266875 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b26caf-3add-4cdb-bb87-df8449e9ec54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:05.279651 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b26caf-3add-4cdb-bb87-df8449e9ec54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:05.281133 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b26caf-3add-4cdb-bb87-df8449e9ec54 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:05.291440 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1122/2241] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:44:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:44:05.574517 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a21b97-b1c0-4dec-a2db-4f77c720baab None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:05.576061 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-60a21b97-b1c0-4dec-a2db-4f77c720baab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:05.590884 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a21b97-b1c0-4dec-a2db-4f77c720baab None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:05.596287 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11e46c6-94c8-4172-a33c-91b0293466e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:05.597168 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11e46c6-94c8-4172-a33c-91b0293466e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:05.597692 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11e46c6-94c8-4172-a33c-91b0293466e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:05.598111 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11e46c6-94c8-4172-a33c-91b0293466e6 admin admin] PATH_INFO: `/v3/projects/43bcfda94ed74808bb64e062f0fdc93f` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:05.599878 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11e46c6-94c8-4172-a33c-91b0293466e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43bcfda94ed74808bb64e062f0fdc93f)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:05.603939 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11e46c6-94c8-4172-a33c-91b0293466e6 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:05.652990 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c11e46c6-94c8-4172-a33c-91b0293466e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43bcfda94ed74808bb64e062f0fdc93f', 'request_id': 'req-c11e46c6-94c8-4172-a33c-91b0293466e6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:05.659840 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c11e46c6-94c8-4172-a33c-91b0293466e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43bcfda94ed74808bb64e062f0fdc93f', 'request_id': 'req-c11e46c6-94c8-4172-a33c-91b0293466e6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:05.666126 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c11e46c6-94c8-4172-a33c-91b0293466e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43bcfda94ed74808bb64e062f0fdc93f', 'request_id': 'req-c11e46c6-94c8-4172-a33c-91b0293466e6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:05.678892 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c11e46c6-94c8-4172-a33c-91b0293466e6 admin admin] The token cache is being invalidate because project 43bcfda94ed74808bb64e062f0fdc93f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:44:05.679261 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c11e46c6-94c8-4172-a33c-91b0293466e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:05.679935 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c11e46c6-94c8-4172-a33c-91b0293466e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:05.682118 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1120/2242] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:44:05 2022] DELETE /identity/v3/projects/43bcfda94ed74808bb64e062f0fdc93f => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:44:05.686370 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b26caf-3add-4cdb-bb87-df8449e9ec54 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:05.696039 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b26caf-3add-4cdb-bb87-df8449e9ec54 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:05.755763 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a75aab2-bda0-4c68-bb0d-ba7802d23ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:05.756425 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a75aab2-bda0-4c68-bb0d-ba7802d23ffd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:05.756744 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a75aab2-bda0-4c68-bb0d-ba7802d23ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:05.757132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a75aab2-bda0-4c68-bb0d-ba7802d23ffd admin admin] PATH_INFO: `/v3/users/6550536c78f74d4cb1e7d7247663b644` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:05.758245 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a75aab2-bda0-4c68-bb0d-ba7802d23ffd admin admin] RBAC: Authorizing `identity:delete_user(user_id=6550536c78f74d4cb1e7d7247663b644)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:05.760226 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a75aab2-bda0-4c68-bb0d-ba7802d23ffd admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:05.807047 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-1a75aab2-bda0-4c68-bb0d-ba7802d23ffd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6550536c78f74d4cb1e7d7247663b644'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:05.813946 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-1a75aab2-bda0-4c68-bb0d-ba7802d23ffd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6550536c78f74d4cb1e7d7247663b644'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:05.826490 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-1a75aab2-bda0-4c68-bb0d-ba7802d23ffd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6550536c78f74d4cb1e7d7247663b644'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:05.833408 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-1a75aab2-bda0-4c68-bb0d-ba7802d23ffd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6550536c78f74d4cb1e7d7247663b644'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:05.834179 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-1a75aab2-bda0-4c68-bb0d-ba7802d23ffd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6550536c78f74d4cb1e7d7247663b644'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:05.840868 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1123/2243] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:44:05 2022] DELETE /identity/v3/users/6550536c78f74d4cb1e7d7247663b644 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:44:06.210666 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11e46c6-94c8-4172-a33c-91b0293466e6 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:06.213024 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c11e46c6-94c8-4172-a33c-91b0293466e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:06.234497 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11e46c6-94c8-4172-a33c-91b0293466e6 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:06.292002 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:06.318016 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06908fae-38f5-4ef8-a00a-37d45d168845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:06.318648 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06908fae-38f5-4ef8-a00a-37d45d168845 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:06.318919 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06908fae-38f5-4ef8-a00a-37d45d168845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:06.319185 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06908fae-38f5-4ef8-a00a-37d45d168845 admin admin] PATH_INFO: `/v3/projects/efc4f25284d54c0fb42ecdb1ae3199a6` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:06.320842 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06908fae-38f5-4ef8-a00a-37d45d168845 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efc4f25284d54c0fb42ecdb1ae3199a6)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:06.323550 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06908fae-38f5-4ef8-a00a-37d45d168845 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:06.378480 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-06908fae-38f5-4ef8-a00a-37d45d168845 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efc4f25284d54c0fb42ecdb1ae3199a6', 'request_id': 'req-06908fae-38f5-4ef8-a00a-37d45d168845'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:06.386370 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-06908fae-38f5-4ef8-a00a-37d45d168845 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efc4f25284d54c0fb42ecdb1ae3199a6', 'request_id': 'req-06908fae-38f5-4ef8-a00a-37d45d168845'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:06.394200 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-06908fae-38f5-4ef8-a00a-37d45d168845 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efc4f25284d54c0fb42ecdb1ae3199a6', 'request_id': 'req-06908fae-38f5-4ef8-a00a-37d45d168845'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:06.409344 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-06908fae-38f5-4ef8-a00a-37d45d168845 admin admin] The token cache is being invalidate because project efc4f25284d54c0fb42ecdb1ae3199a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:44:06.409674 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-06908fae-38f5-4ef8-a00a-37d45d168845 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:06.410287 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-06908fae-38f5-4ef8-a00a-37d45d168845 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:06.412027 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1121/2244] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:44:06 2022] DELETE /identity/v3/projects/efc4f25284d54c0fb42ecdb1ae3199a6 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:44:06.417212 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef83265-b100-4bcd-9d4a-12e8c9e4a9fd None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:06.417745 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef83265-b100-4bcd-9d4a-12e8c9e4a9fd None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:06.418151 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef83265-b100-4bcd-9d4a-12e8c9e4a9fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:06.446984 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-bef83265-b100-4bcd-9d4a-12e8c9e4a9fd None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:06.449451 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-bef83265-b100-4bcd-9d4a-12e8c9e4a9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:06.522374 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1124/2245] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:44:06.530748 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a922dbd0-bba0-4198-90a3-875c03417ac7 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:06.531121 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a922dbd0-bba0-4198-90a3-875c03417ac7 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:06.531320 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a922dbd0-bba0-4198-90a3-875c03417ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:06.554048 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a922dbd0-bba0-4198-90a3-875c03417ac7 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:06.555191 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a922dbd0-bba0-4198-90a3-875c03417ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:06.567750 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1122/2246] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:44:06.571198 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef83265-b100-4bcd-9d4a-12e8c9e4a9fd None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:06.582212 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef83265-b100-4bcd-9d4a-12e8c9e4a9fd None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:06.587525 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e492449-a0d6-4b48-9d6e-434a4741c23d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:06.588132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e492449-a0d6-4b48-9d6e-434a4741c23d admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:06.588409 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e492449-a0d6-4b48-9d6e-434a4741c23d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:06.588727 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e492449-a0d6-4b48-9d6e-434a4741c23d admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:06.589372 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e492449-a0d6-4b48-9d6e-434a4741c23d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:44:06.589672 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e492449-a0d6-4b48-9d6e-434a4741c23d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:06.591267 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e492449-a0d6-4b48-9d6e-434a4741c23d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:06.595829 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:06.612228 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a922dbd0-bba0-4198-90a3-875c03417ac7 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:06.616866 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1125/2247] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:44:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:44:06.624660 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a922dbd0-bba0-4198-90a3-875c03417ac7 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:06.631743 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f3868-bd80-4454-9839-8546585e2056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:06.632486 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f3868-bd80-4454-9839-8546585e2056 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:06.632898 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f3868-bd80-4454-9839-8546585e2056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:06.633262 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f3868-bd80-4454-9839-8546585e2056 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:06.635085 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599f3868-bd80-4454-9839-8546585e2056 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:06.637996 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599f3868-bd80-4454-9839-8546585e2056 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:06.674524 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1123/2248] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:44:06 2022] POST /identity/v3/projects => generated 387 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:44:06.678745 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e492449-a0d6-4b48-9d6e-434a4741c23d admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:06.690625 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e492449-a0d6-4b48-9d6e-434a4741c23d admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:06.760145 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1697d9-e90d-4c94-9c20-7a36783444d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:06.760973 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1697d9-e90d-4c94-9c20-7a36783444d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:06.761354 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1697d9-e90d-4c94-9c20-7a36783444d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:06.761726 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1697d9-e90d-4c94-9c20-7a36783444d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:06.762626 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1697d9-e90d-4c94-9c20-7a36783444d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:06.765236 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1697d9-e90d-4c94-9c20-7a36783444d4 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:06.792752 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1126/2249] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:44:06 2022] POST /identity/v3/users => generated 294 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:44:06.797926 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f3868-bd80-4454-9839-8546585e2056 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:06.811170 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f3868-bd80-4454-9839-8546585e2056 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:06.817632 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5ffbfd-b8be-4850-abbd-cbbd4be661c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:06.818284 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ffbfd-b8be-4850-abbd-cbbd4be661c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:06.818565 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ffbfd-b8be-4850-abbd-cbbd4be661c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:06.819623 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ffbfd-b8be-4850-abbd-cbbd4be661c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:06.820291 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5ffbfd-b8be-4850-abbd-cbbd4be661c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:06.822533 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5ffbfd-b8be-4850-abbd-cbbd4be661c4 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:06.835695 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1124/2250] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:44:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:44:06.839720 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1697d9-e90d-4c94-9c20-7a36783444d4 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:06.850983 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1697d9-e90d-4c94-9c20-7a36783444d4 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:06.856827 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fd8237-2300-4887-a9c1-82fd69fbc603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:06.857452 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd8237-2300-4887-a9c1-82fd69fbc603 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:06.857732 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd8237-2300-4887-a9c1-82fd69fbc603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:06.858007 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd8237-2300-4887-a9c1-82fd69fbc603 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:06.859158 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fd8237-2300-4887-a9c1-82fd69fbc603 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:06.860407 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fd8237-2300-4887-a9c1-82fd69fbc603 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:06.875728 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1127/2251] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:44:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 09:44:06.879600 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5ffbfd-b8be-4850-abbd-cbbd4be661c4 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:06.893400 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5ffbfd-b8be-4850-abbd-cbbd4be661c4 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:06.899100 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4905d945-e058-4646-969b-6ec18afe64c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:06.899839 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905d945-e058-4646-969b-6ec18afe64c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:06.900878 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905d945-e058-4646-969b-6ec18afe64c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:06.901293 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905d945-e058-4646-969b-6ec18afe64c7 admin admin] PATH_INFO: `/v3/projects/1e16dafe8af943a7ab1c040fc940ab9c/users/98a2ca6480e941d68e864992eaa255ca/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:06.920916 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4905d945-e058-4646-969b-6ec18afe64c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e16dafe8af943a7ab1c040fc940ab9c, user_id=98a2ca6480e941d68e864992eaa255ca, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:06.925311 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4905d945-e058-4646-969b-6ec18afe64c7 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:06.950540 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1125/2252] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:44:06 2022] PUT /identity/v3/projects/1e16dafe8af943a7ab1c040fc940ab9c/users/98a2ca6480e941d68e864992eaa255ca/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:44:06.961260 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b62ac3-0f92-4980-b36b-0f6a31454f4d None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:06.961603 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b62ac3-0f92-4980-b36b-0f6a31454f4d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:06.961887 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b62ac3-0f92-4980-b36b-0f6a31454f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:06.998475 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-48b62ac3-0f92-4980-b36b-0f6a31454f4d None None] MFA Rules not processed for user `98a2ca6480e941d68e864992eaa255ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:07.000278 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-48b62ac3-0f92-4980-b36b-0f6a31454f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:07.098064 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1128/2253] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:06 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:44:07.103089 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4905d945-e058-4646-969b-6ec18afe64c7 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:07.204914 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4905d945-e058-4646-969b-6ec18afe64c7 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:07.209944 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a66fe-1b51-4403-9d4d-a97e4a834984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:07.210663 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a66fe-1b51-4403-9d4d-a97e4a834984 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:07.211044 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a66fe-1b51-4403-9d4d-a97e4a834984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:07.211600 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a66fe-1b51-4403-9d4d-a97e4a834984 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:07.212157 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8a66fe-1b51-4403-9d4d-a97e4a834984 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:07.214386 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8a66fe-1b51-4403-9d4d-a97e4a834984 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:07.237086 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:07.247838 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1126/2254] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:44:07 2022] POST /identity/v3/projects => generated 385 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:44:07.252191 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b62ac3-0f92-4980-b36b-0f6a31454f4d None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:07.263984 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b62ac3-0f92-4980-b36b-0f6a31454f4d None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:07.337872 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bff069-c168-4866-9471-b6fb41e96b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:07.339107 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bff069-c168-4866-9471-b6fb41e96b75 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:07.339383 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bff069-c168-4866-9471-b6fb41e96b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:07.339647 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bff069-c168-4866-9471-b6fb41e96b75 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:07.340273 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bff069-c168-4866-9471-b6fb41e96b75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:07.342496 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bff069-c168-4866-9471-b6fb41e96b75 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:07.352637 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:07.369503 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1129/2255] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:44:07 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:44:07.370985 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a66fe-1b51-4403-9d4d-a97e4a834984 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:07.381099 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a66fe-1b51-4403-9d4d-a97e4a834984 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:07.386230 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcf19df-b546-4606-b639-dd3dfd1849ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:07.386819 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf19df-b546-4606-b639-dd3dfd1849ac admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:07.387094 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf19df-b546-4606-b639-dd3dfd1849ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:07.387358 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf19df-b546-4606-b639-dd3dfd1849ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:07.387980 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcf19df-b546-4606-b639-dd3dfd1849ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:07.389679 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcf19df-b546-4606-b639-dd3dfd1849ac admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:07.400700 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1127/2256] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:44:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:44:07.404698 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bff069-c168-4866-9471-b6fb41e96b75 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:07.415718 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bff069-c168-4866-9471-b6fb41e96b75 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:07.420201 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6212220b-5261-4a78-a735-c2c6354316ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:07.420966 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6212220b-5261-4a78-a735-c2c6354316ad admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:07.421239 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6212220b-5261-4a78-a735-c2c6354316ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:07.421613 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6212220b-5261-4a78-a735-c2c6354316ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:07.422275 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6212220b-5261-4a78-a735-c2c6354316ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:07.423830 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6212220b-5261-4a78-a735-c2c6354316ad admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:07.435310 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1130/2257] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:44:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:44:07.439760 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcf19df-b546-4606-b639-dd3dfd1849ac admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:07.451254 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcf19df-b546-4606-b639-dd3dfd1849ac admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:07.456038 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b618f72-7f30-4693-9e83-79a488e1a149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:07.456733 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b618f72-7f30-4693-9e83-79a488e1a149 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:07.457008 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b618f72-7f30-4693-9e83-79a488e1a149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:07.457277 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b618f72-7f30-4693-9e83-79a488e1a149 admin admin] PATH_INFO: `/v3/projects/041b364a74e34b4abc9b03db797bfcd2/users/5b5df3590fe04bfa8a608c53d5154f39/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:07.473141 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b618f72-7f30-4693-9e83-79a488e1a149 admin admin] RBAC: Authorizing `identity:create_grant(project_id=041b364a74e34b4abc9b03db797bfcd2, user_id=5b5df3590fe04bfa8a608c53d5154f39, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:07.475908 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b618f72-7f30-4693-9e83-79a488e1a149 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:07.490984 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1128/2258] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:44:07 2022] PUT /identity/v3/projects/041b364a74e34b4abc9b03db797bfcd2/users/5b5df3590fe04bfa8a608c53d5154f39/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:44:07.504655 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c369f1-bc84-45cb-a5f1-16d65010c43c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:07.504938 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c369f1-bc84-45cb-a5f1-16d65010c43c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:07.505207 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c369f1-bc84-45cb-a5f1-16d65010c43c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:07.536721 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-50c369f1-bc84-45cb-a5f1-16d65010c43c None None] MFA Rules not processed for user `5b5df3590fe04bfa8a608c53d5154f39`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:07.538098 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-50c369f1-bc84-45cb-a5f1-16d65010c43c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:07.617792 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1131/2259] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:44:07.622321 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b618f72-7f30-4693-9e83-79a488e1a149 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:07.694762 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b618f72-7f30-4693-9e83-79a488e1a149 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:07.700274 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6214e5-bcc3-430b-8636-e015f4d3cf4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:07.700907 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6214e5-bcc3-430b-8636-e015f4d3cf4c admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:07.701186 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6214e5-bcc3-430b-8636-e015f4d3cf4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:07.701470 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6214e5-bcc3-430b-8636-e015f4d3cf4c admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:07.702082 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6214e5-bcc3-430b-8636-e015f4d3cf4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:07.704018 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6214e5-bcc3-430b-8636-e015f4d3cf4c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:07.735668 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1129/2260] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:44:07 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:44:07.737074 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c369f1-bc84-45cb-a5f1-16d65010c43c None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:07.747938 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c369f1-bc84-45cb-a5f1-16d65010c43c None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:07.816061 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6271518c-3226-4ca2-9187-0c7fe50dbc27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:07.817121 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6271518c-3226-4ca2-9187-0c7fe50dbc27 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:07.817666 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6271518c-3226-4ca2-9187-0c7fe50dbc27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:07.818179 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6271518c-3226-4ca2-9187-0c7fe50dbc27 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:07.819375 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6271518c-3226-4ca2-9187-0c7fe50dbc27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:07.822834 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6271518c-3226-4ca2-9187-0c7fe50dbc27 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:07.854074 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1132/2261] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:44:07 2022] POST /identity/v3/users => generated 299 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:44:07.855461 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6214e5-bcc3-430b-8636-e015f4d3cf4c admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:07.866680 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6214e5-bcc3-430b-8636-e015f4d3cf4c admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:07.871781 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19736c34-5423-4d89-adf8-78cb33810b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:07.872399 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19736c34-5423-4d89-adf8-78cb33810b77 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:07.872710 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19736c34-5423-4d89-adf8-78cb33810b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:07.873011 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19736c34-5423-4d89-adf8-78cb33810b77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:07.873656 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19736c34-5423-4d89-adf8-78cb33810b77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:07.875252 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19736c34-5423-4d89-adf8-78cb33810b77 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:07.887628 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1130/2262] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:44:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 09:44:07.892659 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6271518c-3226-4ca2-9187-0c7fe50dbc27 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:07.903400 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6271518c-3226-4ca2-9187-0c7fe50dbc27 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:07.908041 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c5306-acb1-4791-92eb-4fd54e39bce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:07.908751 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c5306-acb1-4791-92eb-4fd54e39bce9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:07.909035 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c5306-acb1-4791-92eb-4fd54e39bce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:07.909310 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c5306-acb1-4791-92eb-4fd54e39bce9 admin admin] PATH_INFO: `/v3/projects/859dc2e2fcd5494ab2ba19dea9c8a14e/users/bfaf26677e6f4ce189739a1c2c853139/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:07.927461 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c5306-acb1-4791-92eb-4fd54e39bce9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=859dc2e2fcd5494ab2ba19dea9c8a14e, user_id=bfaf26677e6f4ce189739a1c2c853139, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:07.930191 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c5306-acb1-4791-92eb-4fd54e39bce9 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:07.952446 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1133/2263] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:44:07 2022] PUT /identity/v3/projects/859dc2e2fcd5494ab2ba19dea9c8a14e/users/bfaf26677e6f4ce189739a1c2c853139/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:44:07.956215 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19736c34-5423-4d89-adf8-78cb33810b77 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:07.978409 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19736c34-5423-4d89-adf8-78cb33810b77 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:08.083134 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c30669-0e24-4c8b-8d6d-363b2d18aed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:08.084056 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c30669-0e24-4c8b-8d6d-363b2d18aed8 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:08.084346 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c30669-0e24-4c8b-8d6d-363b2d18aed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:08.084656 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c30669-0e24-4c8b-8d6d-363b2d18aed8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:08.085311 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c30669-0e24-4c8b-8d6d-363b2d18aed8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:08.087956 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c30669-0e24-4c8b-8d6d-363b2d18aed8 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:08.101944 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1131/2264] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:44:07 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 09:44:08.106065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c5306-acb1-4791-92eb-4fd54e39bce9 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:08.122466 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c5306-acb1-4791-92eb-4fd54e39bce9 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:08.130865 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25da8935-f04e-46fa-9cde-cebdf5dc91c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:08.131585 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25da8935-f04e-46fa-9cde-cebdf5dc91c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:08.131986 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25da8935-f04e-46fa-9cde-cebdf5dc91c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:08.132335 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25da8935-f04e-46fa-9cde-cebdf5dc91c9 admin admin] PATH_INFO: `/v3/projects/859dc2e2fcd5494ab2ba19dea9c8a14e/users/bfaf26677e6f4ce189739a1c2c853139/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:08.135870 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25da8935-f04e-46fa-9cde-cebdf5dc91c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=859dc2e2fcd5494ab2ba19dea9c8a14e, user_id=bfaf26677e6f4ce189739a1c2c853139, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:08.139129 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25da8935-f04e-46fa-9cde-cebdf5dc91c9 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:08.155238 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1134/2265] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:44:08 2022] PUT /identity/v3/projects/859dc2e2fcd5494ab2ba19dea9c8a14e/users/bfaf26677e6f4ce189739a1c2c853139/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:44:08.166549 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc87f6-e19b-4dec-8796-ee59628c0077 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:08.166926 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc87f6-e19b-4dec-8796-ee59628c0077 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:08.167278 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc87f6-e19b-4dec-8796-ee59628c0077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:08.212134 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-56cc87f6-e19b-4dec-8796-ee59628c0077 None None] MFA Rules not processed for user `bfaf26677e6f4ce189739a1c2c853139`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:08.213423 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-56cc87f6-e19b-4dec-8796-ee59628c0077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:08.241820 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:08.320089 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1132/2266] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:08 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:44:08.332264 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25da8935-f04e-46fa-9cde-cebdf5dc91c9 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:08.334527 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-25da8935-f04e-46fa-9cde-cebdf5dc91c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:08.373464 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:08.436908 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25da8935-f04e-46fa-9cde-cebdf5dc91c9 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:08.444556 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27782473-58ef-42e3-b992-dc5e59c37d33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:08.445237 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27782473-58ef-42e3-b992-dc5e59c37d33 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:08.445952 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27782473-58ef-42e3-b992-dc5e59c37d33 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:08.445952 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27782473-58ef-42e3-b992-dc5e59c37d33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:08.457246 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27782473-58ef-42e3-b992-dc5e59c37d33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:08.458702 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27782473-58ef-42e3-b992-dc5e59c37d33 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:08.469843 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1135/2267] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:44:08 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:44:08.706460 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cc87f6-e19b-4dec-8796-ee59628c0077 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:08.708364 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-56cc87f6-e19b-4dec-8796-ee59628c0077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:08.776936 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cc87f6-e19b-4dec-8796-ee59628c0077 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:08.781511 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5835bb14-5bd1-4231-913c-d0b227b2379d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:08.782145 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5835bb14-5bd1-4231-913c-d0b227b2379d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:08.782428 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5835bb14-5bd1-4231-913c-d0b227b2379d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:08.782699 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5835bb14-5bd1-4231-913c-d0b227b2379d admin admin] PATH_INFO: `/v3/users/0968b48cb99d4168b36a2602bd87f788` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:08.784420 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5835bb14-5bd1-4231-913c-d0b227b2379d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0968b48cb99d4168b36a2602bd87f788)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:08.786380 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5835bb14-5bd1-4231-913c-d0b227b2379d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:08.835440 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5835bb14-5bd1-4231-913c-d0b227b2379d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0968b48cb99d4168b36a2602bd87f788'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:08.842946 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5835bb14-5bd1-4231-913c-d0b227b2379d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0968b48cb99d4168b36a2602bd87f788'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:08.856489 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5835bb14-5bd1-4231-913c-d0b227b2379d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0968b48cb99d4168b36a2602bd87f788'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:08.866150 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5835bb14-5bd1-4231-913c-d0b227b2379d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0968b48cb99d4168b36a2602bd87f788'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:08.867388 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5835bb14-5bd1-4231-913c-d0b227b2379d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0968b48cb99d4168b36a2602bd87f788'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:08.876890 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1133/2268] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:44:08 2022] DELETE /identity/v3/users/0968b48cb99d4168b36a2602bd87f788 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:44:09.025179 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27782473-58ef-42e3-b992-dc5e59c37d33 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:09.027123 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-27782473-58ef-42e3-b992-dc5e59c37d33 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:09.039042 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27782473-58ef-42e3-b992-dc5e59c37d33 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:09.053138 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5835bb14-5bd1-4231-913c-d0b227b2379d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:09.054602 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5835bb14-5bd1-4231-913c-d0b227b2379d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:09.100915 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:09.104245 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daa3118-883c-40eb-b315-df98e44eb570 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:09.105067 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa3118-883c-40eb-b315-df98e44eb570 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:09.105430 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa3118-883c-40eb-b315-df98e44eb570 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:09.105748 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa3118-883c-40eb-b315-df98e44eb570 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:09.107677 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0daa3118-883c-40eb-b315-df98e44eb570 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:09.129369 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5835bb14-5bd1-4231-913c-d0b227b2379d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:09.134707 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3833271-bf26-42fe-8295-7a4f1102baf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:09.135335 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3833271-bf26-42fe-8295-7a4f1102baf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:09.135611 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3833271-bf26-42fe-8295-7a4f1102baf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:09.135878 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3833271-bf26-42fe-8295-7a4f1102baf7 admin admin] PATH_INFO: `/v3/projects/f7b8fb656ee34fae8d20679c0e0f7ff9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:09.137039 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3833271-bf26-42fe-8295-7a4f1102baf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7b8fb656ee34fae8d20679c0e0f7ff9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:09.139141 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3833271-bf26-42fe-8295-7a4f1102baf7 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:09.169105 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daa3118-883c-40eb-b315-df98e44eb570 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:09.170526 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daa3118-883c-40eb-b315-df98e44eb570 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:09.179523 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1136/2269] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:44:09 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:44:09.186265 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b3833271-bf26-42fe-8295-7a4f1102baf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7b8fb656ee34fae8d20679c0e0f7ff9', 'request_id': 'req-b3833271-bf26-42fe-8295-7a4f1102baf7'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:09.192609 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b3833271-bf26-42fe-8295-7a4f1102baf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7b8fb656ee34fae8d20679c0e0f7ff9', 'request_id': 'req-b3833271-bf26-42fe-8295-7a4f1102baf7'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:09.199585 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b3833271-bf26-42fe-8295-7a4f1102baf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7b8fb656ee34fae8d20679c0e0f7ff9', 'request_id': 'req-b3833271-bf26-42fe-8295-7a4f1102baf7'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:09.216553 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b3833271-bf26-42fe-8295-7a4f1102baf7 admin admin] The token cache is being invalidate because project f7b8fb656ee34fae8d20679c0e0f7ff9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:44:09.217995 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b3833271-bf26-42fe-8295-7a4f1102baf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:09.218826 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b3833271-bf26-42fe-8295-7a4f1102baf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:09.220727 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1134/2270] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:44:09 2022] DELETE /identity/v3/projects/f7b8fb656ee34fae8d20679c0e0f7ff9 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:44:09.224593 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daa3118-883c-40eb-b315-df98e44eb570 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:09.237430 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daa3118-883c-40eb-b315-df98e44eb570 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:09.303312 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa74e8-1018-4dec-9a12-9edbc0e187d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:09.304016 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa74e8-1018-4dec-9a12-9edbc0e187d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:09.304298 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa74e8-1018-4dec-9a12-9edbc0e187d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:09.304604 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa74e8-1018-4dec-9a12-9edbc0e187d5 admin admin] PATH_INFO: `/v3/users/655a2d89f6e440bc8634c5d8aab17ccf` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:09.306321 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaa74e8-1018-4dec-9a12-9edbc0e187d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=655a2d89f6e440bc8634c5d8aab17ccf)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:09.308232 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaa74e8-1018-4dec-9a12-9edbc0e187d5 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:09.344659 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252c9991-15f5-4353-a9f5-4c6041c6ba30 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:09.344938 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252c9991-15f5-4353-a9f5-4c6041c6ba30 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:09.345381 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252c9991-15f5-4353-a9f5-4c6041c6ba30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:09.350474 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:09.366446 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-252c9991-15f5-4353-a9f5-4c6041c6ba30 None None] MFA Rules not processed for user `127c4fdd8f2b40ae8d98208d64a7f20d`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:09.367650 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-252c9991-15f5-4353-a9f5-4c6041c6ba30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:09.381136 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ebaa74e8-1018-4dec-9a12-9edbc0e187d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '655a2d89f6e440bc8634c5d8aab17ccf'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:09.389459 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ebaa74e8-1018-4dec-9a12-9edbc0e187d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '655a2d89f6e440bc8634c5d8aab17ccf'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:09.402224 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ebaa74e8-1018-4dec-9a12-9edbc0e187d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '655a2d89f6e440bc8634c5d8aab17ccf'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:09.413703 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ebaa74e8-1018-4dec-9a12-9edbc0e187d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '655a2d89f6e440bc8634c5d8aab17ccf'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:09.414395 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ebaa74e8-1018-4dec-9a12-9edbc0e187d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '655a2d89f6e440bc8634c5d8aab17ccf'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:09.422550 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1137/2271] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:44:09 2022] DELETE /identity/v3/users/655a2d89f6e440bc8634c5d8aab17ccf => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:44:09.437943 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1135/2272] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:44:09.681731 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa74e8-1018-4dec-9a12-9edbc0e187d5 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:09.683175 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ebaa74e8-1018-4dec-9a12-9edbc0e187d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:09.760133 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa74e8-1018-4dec-9a12-9edbc0e187d5 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:09.764837 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea0f8d-5ed9-4683-bd24-9be6272257e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:09.765457 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea0f8d-5ed9-4683-bd24-9be6272257e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:09.765736 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea0f8d-5ed9-4683-bd24-9be6272257e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:09.766083 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea0f8d-5ed9-4683-bd24-9be6272257e8 admin admin] PATH_INFO: `/v3/projects/d6bd77f2911f4d7fb339ce7d80cf4094` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:09.767099 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abea0f8d-5ed9-4683-bd24-9be6272257e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6bd77f2911f4d7fb339ce7d80cf4094)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:09.769014 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abea0f8d-5ed9-4683-bd24-9be6272257e8 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:09.818386 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-abea0f8d-5ed9-4683-bd24-9be6272257e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6bd77f2911f4d7fb339ce7d80cf4094', 'request_id': 'req-abea0f8d-5ed9-4683-bd24-9be6272257e8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:09.826115 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-abea0f8d-5ed9-4683-bd24-9be6272257e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6bd77f2911f4d7fb339ce7d80cf4094', 'request_id': 'req-abea0f8d-5ed9-4683-bd24-9be6272257e8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:09.832923 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-abea0f8d-5ed9-4683-bd24-9be6272257e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6bd77f2911f4d7fb339ce7d80cf4094', 'request_id': 'req-abea0f8d-5ed9-4683-bd24-9be6272257e8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:09.845585 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-abea0f8d-5ed9-4683-bd24-9be6272257e8 admin admin] The token cache is being invalidate because project d6bd77f2911f4d7fb339ce7d80cf4094 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:44:09.845877 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-abea0f8d-5ed9-4683-bd24-9be6272257e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:09.846521 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-abea0f8d-5ed9-4683-bd24-9be6272257e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:09.848272 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1138/2273] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:44:09 2022] DELETE /identity/v3/projects/d6bd77f2911f4d7fb339ce7d80cf4094 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:44:13.304329 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252c9991-15f5-4353-a9f5-4c6041c6ba30 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:13.305897 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-252c9991-15f5-4353-a9f5-4c6041c6ba30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:13.364442 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252c9991-15f5-4353-a9f5-4c6041c6ba30 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:13.388334 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:13.415252 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0a948f-8f45-4d42-9310-0813f85e1530 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:13.415850 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a948f-8f45-4d42-9310-0813f85e1530 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:13.416121 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a948f-8f45-4d42-9310-0813f85e1530 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:13.416413 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a948f-8f45-4d42-9310-0813f85e1530 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:13.418154 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0a0a948f-8f45-4d42-9310-0813f85e1530 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:13.487645 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0a948f-8f45-4d42-9310-0813f85e1530 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:13.489343 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0a948f-8f45-4d42-9310-0813f85e1530 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:13.501593 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1136/2274] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:44:13 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:44:15.660115 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea0f8d-5ed9-4683-bd24-9be6272257e8 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:15.662298 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-abea0f8d-5ed9-4683-bd24-9be6272257e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:15.701743 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:15.734939 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea0f8d-5ed9-4683-bd24-9be6272257e8 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:15.741950 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be30735-8f5a-47eb-9876-3dd00dfb5e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:15.742970 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be30735-8f5a-47eb-9876-3dd00dfb5e32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:15.743418 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be30735-8f5a-47eb-9876-3dd00dfb5e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:15.743691 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be30735-8f5a-47eb-9876-3dd00dfb5e32 admin admin] PATH_INFO: `/v3/users/98a2ca6480e941d68e864992eaa255ca` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:15.744880 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be30735-8f5a-47eb-9876-3dd00dfb5e32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98a2ca6480e941d68e864992eaa255ca)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:15.746882 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be30735-8f5a-47eb-9876-3dd00dfb5e32 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:15.823677 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5be30735-8f5a-47eb-9876-3dd00dfb5e32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98a2ca6480e941d68e864992eaa255ca'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:15.830752 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5be30735-8f5a-47eb-9876-3dd00dfb5e32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98a2ca6480e941d68e864992eaa255ca'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:15.844315 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5be30735-8f5a-47eb-9876-3dd00dfb5e32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98a2ca6480e941d68e864992eaa255ca'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:15.852910 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5be30735-8f5a-47eb-9876-3dd00dfb5e32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98a2ca6480e941d68e864992eaa255ca'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:15.854952 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5be30735-8f5a-47eb-9876-3dd00dfb5e32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98a2ca6480e941d68e864992eaa255ca'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:15.866023 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1139/2275] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:44:15 2022] DELETE /identity/v3/users/98a2ca6480e941d68e864992eaa255ca => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:44:15.867691 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8a001-3b63-4470-9551-7cea4a82ed4c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:15.868815 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8a001-3b63-4470-9551-7cea4a82ed4c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:15.869194 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8a001-3b63-4470-9551-7cea4a82ed4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:15.888985 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-49b8a001-3b63-4470-9551-7cea4a82ed4c None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:15.890244 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-49b8a001-3b63-4470-9551-7cea4a82ed4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:15.952144 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1137/2276] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:44:15.962665 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be30735-8f5a-47eb-9876-3dd00dfb5e32 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:15.964133 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5be30735-8f5a-47eb-9876-3dd00dfb5e32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:16.025235 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be30735-8f5a-47eb-9876-3dd00dfb5e32 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:16.029359 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97ecb9a-85fc-4c2b-a821-44f5388874cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:16.030027 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97ecb9a-85fc-4c2b-a821-44f5388874cf service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:16.030342 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97ecb9a-85fc-4c2b-a821-44f5388874cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:16.030677 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97ecb9a-85fc-4c2b-a821-44f5388874cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:16.042248 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97ecb9a-85fc-4c2b-a821-44f5388874cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:16.043709 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97ecb9a-85fc-4c2b-a821-44f5388874cf service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:16.053460 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1140/2277] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:44:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:44:16.338239 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b8a001-3b63-4470-9551-7cea4a82ed4c None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:16.339767 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-49b8a001-3b63-4470-9551-7cea4a82ed4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:16.367358 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b8a001-3b63-4470-9551-7cea4a82ed4c None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:16.374256 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9e1b18-2c29-4145-be07-d9fcaa860b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:16.374782 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e1b18-2c29-4145-be07-d9fcaa860b56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:16.375066 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e1b18-2c29-4145-be07-d9fcaa860b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:16.375339 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e1b18-2c29-4145-be07-d9fcaa860b56 admin admin] PATH_INFO: `/v3/projects/1e16dafe8af943a7ab1c040fc940ab9c` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:16.376495 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9e1b18-2c29-4145-be07-d9fcaa860b56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e16dafe8af943a7ab1c040fc940ab9c)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:16.378585 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9e1b18-2c29-4145-be07-d9fcaa860b56 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:16.429238 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9c9e1b18-2c29-4145-be07-d9fcaa860b56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e16dafe8af943a7ab1c040fc940ab9c', 'request_id': 'req-9c9e1b18-2c29-4145-be07-d9fcaa860b56'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:16.436352 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9c9e1b18-2c29-4145-be07-d9fcaa860b56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e16dafe8af943a7ab1c040fc940ab9c', 'request_id': 'req-9c9e1b18-2c29-4145-be07-d9fcaa860b56'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:16.443150 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9c9e1b18-2c29-4145-be07-d9fcaa860b56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e16dafe8af943a7ab1c040fc940ab9c', 'request_id': 'req-9c9e1b18-2c29-4145-be07-d9fcaa860b56'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:16.459145 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9c9e1b18-2c29-4145-be07-d9fcaa860b56 admin admin] The token cache is being invalidate because project 1e16dafe8af943a7ab1c040fc940ab9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:44:16.459683 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9c9e1b18-2c29-4145-be07-d9fcaa860b56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:16.460314 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9c9e1b18-2c29-4145-be07-d9fcaa860b56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:16.462426 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1138/2278] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:44:16 2022] DELETE /identity/v3/projects/1e16dafe8af943a7ab1c040fc940ab9c => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:44:16.466599 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97ecb9a-85fc-4c2b-a821-44f5388874cf service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:16.476446 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97ecb9a-85fc-4c2b-a821-44f5388874cf service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:16.528777 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:16.539179 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeb2e72-8b4d-4c83-9a4a-3f7e6fb5f1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:16.539787 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb2e72-8b4d-4c83-9a4a-3f7e6fb5f1c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:16.540263 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb2e72-8b4d-4c83-9a4a-3f7e6fb5f1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:16.540719 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb2e72-8b4d-4c83-9a4a-3f7e6fb5f1c5 admin admin] PATH_INFO: `/v3/users/5b5df3590fe04bfa8a608c53d5154f39` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:16.542034 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeb2e72-8b4d-4c83-9a4a-3f7e6fb5f1c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b5df3590fe04bfa8a608c53d5154f39)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:16.543950 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeb2e72-8b4d-4c83-9a4a-3f7e6fb5f1c5 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:16.597000 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9eeb2e72-8b4d-4c83-9a4a-3f7e6fb5f1c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b5df3590fe04bfa8a608c53d5154f39'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:16.604192 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9eeb2e72-8b4d-4c83-9a4a-3f7e6fb5f1c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b5df3590fe04bfa8a608c53d5154f39'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:16.616935 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9eeb2e72-8b4d-4c83-9a4a-3f7e6fb5f1c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b5df3590fe04bfa8a608c53d5154f39'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:16.626187 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9eeb2e72-8b4d-4c83-9a4a-3f7e6fb5f1c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b5df3590fe04bfa8a608c53d5154f39'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:16.626884 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9eeb2e72-8b4d-4c83-9a4a-3f7e6fb5f1c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b5df3590fe04bfa8a608c53d5154f39'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:16.634302 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1141/2279] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:44:16 2022] DELETE /identity/v3/users/5b5df3590fe04bfa8a608c53d5154f39 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:44:17.011410 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9e1b18-2c29-4145-be07-d9fcaa860b56 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:17.014512 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9c9e1b18-2c29-4145-be07-d9fcaa860b56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:17.036254 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:17.047061 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9e1b18-2c29-4145-be07-d9fcaa860b56 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:17.118828 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49315b87-5c00-4a2f-9a7e-1af925f1a62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:17.119634 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49315b87-5c00-4a2f-9a7e-1af925f1a62c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:17.120439 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49315b87-5c00-4a2f-9a7e-1af925f1a62c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:17.120439 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49315b87-5c00-4a2f-9a7e-1af925f1a62c admin admin] PATH_INFO: `/v3/projects/041b364a74e34b4abc9b03db797bfcd2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:17.121678 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49315b87-5c00-4a2f-9a7e-1af925f1a62c admin admin] RBAC: Authorizing `identity:delete_project(project_id=041b364a74e34b4abc9b03db797bfcd2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:17.124498 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49315b87-5c00-4a2f-9a7e-1af925f1a62c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:17.182532 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-49315b87-5c00-4a2f-9a7e-1af925f1a62c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '041b364a74e34b4abc9b03db797bfcd2', 'request_id': 'req-49315b87-5c00-4a2f-9a7e-1af925f1a62c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:17.190019 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-49315b87-5c00-4a2f-9a7e-1af925f1a62c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '041b364a74e34b4abc9b03db797bfcd2', 'request_id': 'req-49315b87-5c00-4a2f-9a7e-1af925f1a62c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:17.196243 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-49315b87-5c00-4a2f-9a7e-1af925f1a62c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '041b364a74e34b4abc9b03db797bfcd2', 'request_id': 'req-49315b87-5c00-4a2f-9a7e-1af925f1a62c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:17.211361 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-49315b87-5c00-4a2f-9a7e-1af925f1a62c admin admin] The token cache is being invalidate because project 041b364a74e34b4abc9b03db797bfcd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:44:17.211692 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-49315b87-5c00-4a2f-9a7e-1af925f1a62c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:17.212329 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-49315b87-5c00-4a2f-9a7e-1af925f1a62c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:17.213893 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1139/2280] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:44:17 2022] DELETE /identity/v3/projects/041b364a74e34b4abc9b03db797bfcd2 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:44:17.218025 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeb2e72-8b4d-4c83-9a4a-3f7e6fb5f1c5 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:17.219541 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9eeb2e72-8b4d-4c83-9a4a-3f7e6fb5f1c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:17.234112 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeb2e72-8b4d-4c83-9a4a-3f7e6fb5f1c5 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:17.302948 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2cf74b-83c7-4983-a40e-8cdf6a01ecc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:17.303521 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2cf74b-83c7-4983-a40e-8cdf6a01ecc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:17.303799 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2cf74b-83c7-4983-a40e-8cdf6a01ecc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:17.304072 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2cf74b-83c7-4983-a40e-8cdf6a01ecc1 admin admin] PATH_INFO: `/v3/users/bfaf26677e6f4ce189739a1c2c853139` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:17.305281 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2cf74b-83c7-4983-a40e-8cdf6a01ecc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfaf26677e6f4ce189739a1c2c853139)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:17.307757 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2cf74b-83c7-4983-a40e-8cdf6a01ecc1 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:17.357599 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0e2cf74b-83c7-4983-a40e-8cdf6a01ecc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfaf26677e6f4ce189739a1c2c853139'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:17.365436 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0e2cf74b-83c7-4983-a40e-8cdf6a01ecc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfaf26677e6f4ce189739a1c2c853139'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:17.378383 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0e2cf74b-83c7-4983-a40e-8cdf6a01ecc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfaf26677e6f4ce189739a1c2c853139'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:17.380058 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:17.386255 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0e2cf74b-83c7-4983-a40e-8cdf6a01ecc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfaf26677e6f4ce189739a1c2c853139'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:17.387147 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0e2cf74b-83c7-4983-a40e-8cdf6a01ecc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfaf26677e6f4ce189739a1c2c853139'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:17.394129 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1142/2281] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:44:17 2022] DELETE /identity/v3/users/bfaf26677e6f4ce189739a1c2c853139 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:44:17.740461 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49315b87-5c00-4a2f-9a7e-1af925f1a62c admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:17.743514 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-49315b87-5c00-4a2f-9a7e-1af925f1a62c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:17.763364 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49315b87-5c00-4a2f-9a7e-1af925f1a62c admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:17.835018 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a089a2f1-aace-4f71-af6f-eb48cfeda2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:17.835654 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a089a2f1-aace-4f71-af6f-eb48cfeda2b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:17.836018 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a089a2f1-aace-4f71-af6f-eb48cfeda2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:17.836189 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a089a2f1-aace-4f71-af6f-eb48cfeda2b8 admin admin] PATH_INFO: `/v3/projects/859dc2e2fcd5494ab2ba19dea9c8a14e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:17.837950 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a089a2f1-aace-4f71-af6f-eb48cfeda2b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=859dc2e2fcd5494ab2ba19dea9c8a14e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:17.839871 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a089a2f1-aace-4f71-af6f-eb48cfeda2b8 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:17.880226 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:17.890972 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a089a2f1-aace-4f71-af6f-eb48cfeda2b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '859dc2e2fcd5494ab2ba19dea9c8a14e', 'request_id': 'req-a089a2f1-aace-4f71-af6f-eb48cfeda2b8'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:17.897779 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a089a2f1-aace-4f71-af6f-eb48cfeda2b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '859dc2e2fcd5494ab2ba19dea9c8a14e', 'request_id': 'req-a089a2f1-aace-4f71-af6f-eb48cfeda2b8'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:17.904437 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a089a2f1-aace-4f71-af6f-eb48cfeda2b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '859dc2e2fcd5494ab2ba19dea9c8a14e', 'request_id': 'req-a089a2f1-aace-4f71-af6f-eb48cfeda2b8'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:17.918396 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a089a2f1-aace-4f71-af6f-eb48cfeda2b8 admin admin] The token cache is being invalidate because project 859dc2e2fcd5494ab2ba19dea9c8a14e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:44:17.918583 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a089a2f1-aace-4f71-af6f-eb48cfeda2b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:17.919308 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a089a2f1-aace-4f71-af6f-eb48cfeda2b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:17.920989 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1140/2282] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:44:17 2022] DELETE /identity/v3/projects/859dc2e2fcd5494ab2ba19dea9c8a14e => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:44:17.927045 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff6fc8-7881-4f42-88d2-69e06fb26955 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:17.927407 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff6fc8-7881-4f42-88d2-69e06fb26955 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:17.927770 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff6fc8-7881-4f42-88d2-69e06fb26955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:17.953161 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-e3ff6fc8-7881-4f42-88d2-69e06fb26955 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:17.954285 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e3ff6fc8-7881-4f42-88d2-69e06fb26955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:18.022507 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1143/2283] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:44:18.032017 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57813ea-be4c-4422-afad-70401b6074e1 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:18.032543 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57813ea-be4c-4422-afad-70401b6074e1 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:18.032946 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57813ea-be4c-4422-afad-70401b6074e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:18.054797 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-e57813ea-be4c-4422-afad-70401b6074e1 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:18.056208 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e57813ea-be4c-4422-afad-70401b6074e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:18.066626 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1141/2284] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:44:18.072281 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ff6fc8-7881-4f42-88d2-69e06fb26955 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:18.096201 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ff6fc8-7881-4f42-88d2-69e06fb26955 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:18.101101 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15613980-0a92-4f00-9b03-c285a1497833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:18.101717 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15613980-0a92-4f00-9b03-c285a1497833 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:18.102490 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15613980-0a92-4f00-9b03-c285a1497833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:18.102490 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15613980-0a92-4f00-9b03-c285a1497833 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:18.102925 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15613980-0a92-4f00-9b03-c285a1497833 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:44:18.103224 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15613980-0a92-4f00-9b03-c285a1497833 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:18.104842 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15613980-0a92-4f00-9b03-c285a1497833 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:18.117712 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1144/2285] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:44:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:44:18.123502 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57813ea-be4c-4422-afad-70401b6074e1 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:18.136442 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57813ea-be4c-4422-afad-70401b6074e1 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:18.141257 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7cf9f2-a483-432d-b7e0-83c52248ea15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:18.141852 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7cf9f2-a483-432d-b7e0-83c52248ea15 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:18.142241 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7cf9f2-a483-432d-b7e0-83c52248ea15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:18.142515 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7cf9f2-a483-432d-b7e0-83c52248ea15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:18.143138 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7cf9f2-a483-432d-b7e0-83c52248ea15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:18.145127 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7cf9f2-a483-432d-b7e0-83c52248ea15 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:18.180471 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1142/2286] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:44:18 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:44:18.186769 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15613980-0a92-4f00-9b03-c285a1497833 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:18.199160 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15613980-0a92-4f00-9b03-c285a1497833 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:18.257386 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50755c2-c45b-4f7a-9c9d-94193883bf06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:18.259282 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50755c2-c45b-4f7a-9c9d-94193883bf06 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:18.259564 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50755c2-c45b-4f7a-9c9d-94193883bf06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:18.259835 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50755c2-c45b-4f7a-9c9d-94193883bf06 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:18.260464 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50755c2-c45b-4f7a-9c9d-94193883bf06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:18.262637 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50755c2-c45b-4f7a-9c9d-94193883bf06 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:18.285191 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1145/2287] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:44:18 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:44:18.289404 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7cf9f2-a483-432d-b7e0-83c52248ea15 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:18.300008 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7cf9f2-a483-432d-b7e0-83c52248ea15 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:18.304713 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ef761-1c4c-4e3e-88c7-1ada57b67275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:18.305852 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ef761-1c4c-4e3e-88c7-1ada57b67275 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:18.306126 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ef761-1c4c-4e3e-88c7-1ada57b67275 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:18.306390 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ef761-1c4c-4e3e-88c7-1ada57b67275 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:18.307022 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ef761-1c4c-4e3e-88c7-1ada57b67275 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:18.308648 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ef761-1c4c-4e3e-88c7-1ada57b67275 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:18.318851 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1143/2288] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:44:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:44:18.323796 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50755c2-c45b-4f7a-9c9d-94193883bf06 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:18.333837 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50755c2-c45b-4f7a-9c9d-94193883bf06 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:18.338634 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4619d9-1d84-4638-87e0-80ccfeb55ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:18.339249 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4619d9-1d84-4638-87e0-80ccfeb55ff7 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:18.339506 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4619d9-1d84-4638-87e0-80ccfeb55ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:18.339776 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4619d9-1d84-4638-87e0-80ccfeb55ff7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:18.340402 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4619d9-1d84-4638-87e0-80ccfeb55ff7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:18.342126 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4619d9-1d84-4638-87e0-80ccfeb55ff7 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:18.352949 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1146/2289] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:44:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:44:18.357432 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ef761-1c4c-4e3e-88c7-1ada57b67275 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:18.367278 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ef761-1c4c-4e3e-88c7-1ada57b67275 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:18.372782 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc80599-bb88-4660-9f29-00a5710eef7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:18.373435 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc80599-bb88-4660-9f29-00a5710eef7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:18.373709 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc80599-bb88-4660-9f29-00a5710eef7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:18.373979 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc80599-bb88-4660-9f29-00a5710eef7c admin admin] PATH_INFO: `/v3/projects/f5b8d4a1ce084cc6b3e84ebb7778e5ae/users/52ac2b5469904e4a8a742de14fcc8508/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:18.389768 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc80599-bb88-4660-9f29-00a5710eef7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5b8d4a1ce084cc6b3e84ebb7778e5ae, user_id=52ac2b5469904e4a8a742de14fcc8508, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:18.392536 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc80599-bb88-4660-9f29-00a5710eef7c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:18.406248 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1144/2290] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:44:18 2022] PUT /identity/v3/projects/f5b8d4a1ce084cc6b3e84ebb7778e5ae/users/52ac2b5469904e4a8a742de14fcc8508/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:44:18.411612 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d422e-981e-4805-a76e-7e41650d0f61 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:18.411898 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d422e-981e-4805-a76e-7e41650d0f61 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:18.412167 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d422e-981e-4805-a76e-7e41650d0f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:18.431685 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f23d422e-981e-4805-a76e-7e41650d0f61 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:18.432969 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f23d422e-981e-4805-a76e-7e41650d0f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:18.483886 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:18.496220 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1147/2291] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:44:18.505252 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc80599-bb88-4660-9f29-00a5710eef7c admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:18.506929 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-bdc80599-bb88-4660-9f29-00a5710eef7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:18.569294 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc80599-bb88-4660-9f29-00a5710eef7c admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:18.582890 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dbca0f-7c1f-41c2-bf36-564e3b1531ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:18.582890 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dbca0f-7c1f-41c2-bf36-564e3b1531ba service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:18.583148 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dbca0f-7c1f-41c2-bf36-564e3b1531ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:18.583479 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dbca0f-7c1f-41c2-bf36-564e3b1531ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:18.593595 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dbca0f-7c1f-41c2-bf36-564e3b1531ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:18.594957 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dbca0f-7c1f-41c2-bf36-564e3b1531ba service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:18.605011 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1145/2292] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:44:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:44:19.351275 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d82c8a-d053-4c13-8b82-0ef4d572d4ce None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:19.351779 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d82c8a-d053-4c13-8b82-0ef4d572d4ce None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:19.351879 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d82c8a-d053-4c13-8b82-0ef4d572d4ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:19.380912 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-79d82c8a-d053-4c13-8b82-0ef4d572d4ce None None] MFA Rules not processed for user `52ac2b5469904e4a8a742de14fcc8508`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:19.382129 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-79d82c8a-d053-4c13-8b82-0ef4d572d4ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:19.449664 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1148/2293] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:44:19.460758 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dbca0f-7c1f-41c2-bf36-564e3b1531ba service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:19.462301 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-80dbca0f-7c1f-41c2-bf36-564e3b1531ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:19.521901 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dbca0f-7c1f-41c2-bf36-564e3b1531ba service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:19.525529 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7714b23-4d98-4630-9a33-ba9a921b6992 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:19.526139 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7714b23-4d98-4630-9a33-ba9a921b6992 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:19.526416 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7714b23-4d98-4630-9a33-ba9a921b6992 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:19.526678 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7714b23-4d98-4630-9a33-ba9a921b6992 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:19.535871 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7714b23-4d98-4630-9a33-ba9a921b6992 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:19.537466 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7714b23-4d98-4630-9a33-ba9a921b6992 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:19.548025 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1146/2294] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:44:19 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:44:20.970023 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0231796-a07a-4756-b22a-20b2526ac7a1 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:20.970479 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0231796-a07a-4756-b22a-20b2526ac7a1 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:20.970607 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0231796-a07a-4756-b22a-20b2526ac7a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:20.990636 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f0231796-a07a-4756-b22a-20b2526ac7a1 None None] MFA Rules not processed for user `52ac2b5469904e4a8a742de14fcc8508`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:20.991815 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f0231796-a07a-4756-b22a-20b2526ac7a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:20.998528 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1149/2295] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:44:21.120115 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7714b23-4d98-4630-9a33-ba9a921b6992 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:21.121546 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d7714b23-4d98-4630-9a33-ba9a921b6992 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:21.134002 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7714b23-4d98-4630-9a33-ba9a921b6992 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:21.137568 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a503876-238d-41ee-91a6-8ef96486a468 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:21.138368 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a503876-238d-41ee-91a6-8ef96486a468 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:21.138646 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a503876-238d-41ee-91a6-8ef96486a468 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:21.138920 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a503876-238d-41ee-91a6-8ef96486a468 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:21.140624 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5a503876-238d-41ee-91a6-8ef96486a468 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:21.149524 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:21.154397 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a503876-238d-41ee-91a6-8ef96486a468 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:21.156356 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a503876-238d-41ee-91a6-8ef96486a468 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:21.168099 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1147/2296] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:44:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:44:24.993867 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0231796-a07a-4756-b22a-20b2526ac7a1 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:24.995427 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f0231796-a07a-4756-b22a-20b2526ac7a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:25.009329 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0231796-a07a-4756-b22a-20b2526ac7a1 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:25.013848 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95dcb19-537a-4d2f-9d0a-c17cbaf8809c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:25.014470 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95dcb19-537a-4d2f-9d0a-c17cbaf8809c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:25.014742 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95dcb19-537a-4d2f-9d0a-c17cbaf8809c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:25.015198 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95dcb19-537a-4d2f-9d0a-c17cbaf8809c admin admin] PATH_INFO: `/v3/users/2dea51c29cbd46e8bd1454e10eaac012` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:25.016313 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95dcb19-537a-4d2f-9d0a-c17cbaf8809c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dea51c29cbd46e8bd1454e10eaac012)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:25.018229 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95dcb19-537a-4d2f-9d0a-c17cbaf8809c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:25.068362 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a95dcb19-537a-4d2f-9d0a-c17cbaf8809c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dea51c29cbd46e8bd1454e10eaac012'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:25.076334 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a95dcb19-537a-4d2f-9d0a-c17cbaf8809c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dea51c29cbd46e8bd1454e10eaac012'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:25.090002 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a95dcb19-537a-4d2f-9d0a-c17cbaf8809c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dea51c29cbd46e8bd1454e10eaac012'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:25.098551 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a95dcb19-537a-4d2f-9d0a-c17cbaf8809c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dea51c29cbd46e8bd1454e10eaac012'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:25.099189 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a95dcb19-537a-4d2f-9d0a-c17cbaf8809c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dea51c29cbd46e8bd1454e10eaac012'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:25.109242 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1150/2297] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:44:24 2022] DELETE /identity/v3/users/2dea51c29cbd46e8bd1454e10eaac012 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:44:25.316926 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a503876-238d-41ee-91a6-8ef96486a468 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:25.329243 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a503876-238d-41ee-91a6-8ef96486a468 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:25.407672 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1da289-3590-490b-a70b-12574b13efe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:25.408310 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1da289-3590-490b-a70b-12574b13efe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:25.408613 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1da289-3590-490b-a70b-12574b13efe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:25.408885 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1da289-3590-490b-a70b-12574b13efe2 admin admin] PATH_INFO: `/v3/projects/7295248456c446bbb3f7d2d5953c9360` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:25.409997 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1da289-3590-490b-a70b-12574b13efe2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7295248456c446bbb3f7d2d5953c9360)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:25.412063 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1da289-3590-490b-a70b-12574b13efe2 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:25.505111 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6e1da289-3590-490b-a70b-12574b13efe2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7295248456c446bbb3f7d2d5953c9360', 'request_id': 'req-6e1da289-3590-490b-a70b-12574b13efe2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:25.512040 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6e1da289-3590-490b-a70b-12574b13efe2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7295248456c446bbb3f7d2d5953c9360', 'request_id': 'req-6e1da289-3590-490b-a70b-12574b13efe2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:25.519156 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6e1da289-3590-490b-a70b-12574b13efe2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7295248456c446bbb3f7d2d5953c9360', 'request_id': 'req-6e1da289-3590-490b-a70b-12574b13efe2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:25.532714 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6e1da289-3590-490b-a70b-12574b13efe2 admin admin] The token cache is being invalidate because project 7295248456c446bbb3f7d2d5953c9360 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:44:25.533013 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6e1da289-3590-490b-a70b-12574b13efe2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:25.533626 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6e1da289-3590-490b-a70b-12574b13efe2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:25.535122 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1148/2298] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:44:25 2022] DELETE /identity/v3/projects/7295248456c446bbb3f7d2d5953c9360 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:44:25.540160 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5fc14a-6751-4fee-80ce-a152bb645ee1 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:25.540390 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5fc14a-6751-4fee-80ce-a152bb645ee1 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:25.540704 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5fc14a-6751-4fee-80ce-a152bb645ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:25.561222 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-aa5fc14a-6751-4fee-80ce-a152bb645ee1 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:25.562542 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-aa5fc14a-6751-4fee-80ce-a152bb645ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:25.612109 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:25.627480 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1151/2299] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:44:25.635771 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68faa3d3-0158-42aa-b41b-bd1488baedaf None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:25.636127 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68faa3d3-0158-42aa-b41b-bd1488baedaf None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:25.636472 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68faa3d3-0158-42aa-b41b-bd1488baedaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:25.656789 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-68faa3d3-0158-42aa-b41b-bd1488baedaf None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:25.658088 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-68faa3d3-0158-42aa-b41b-bd1488baedaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:25.666285 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1149/2300] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:44:25.670565 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5fc14a-6751-4fee-80ce-a152bb645ee1 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:25.680305 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5fc14a-6751-4fee-80ce-a152bb645ee1 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:25.685401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8bd69d-0092-4ed1-9f6c-70e7cd0aef1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:25.686006 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8bd69d-0092-4ed1-9f6c-70e7cd0aef1f admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:25.686284 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8bd69d-0092-4ed1-9f6c-70e7cd0aef1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:25.686551 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8bd69d-0092-4ed1-9f6c-70e7cd0aef1f admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:25.687215 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8bd69d-0092-4ed1-9f6c-70e7cd0aef1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:44:25.687519 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8bd69d-0092-4ed1-9f6c-70e7cd0aef1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:25.689165 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8bd69d-0092-4ed1-9f6c-70e7cd0aef1f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:25.705216 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1152/2301] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:44:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:44:25.706613 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68faa3d3-0158-42aa-b41b-bd1488baedaf None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:25.716702 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68faa3d3-0158-42aa-b41b-bd1488baedaf None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:25.721667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2908d6-7c6b-40c9-aaf3-4088512812e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:25.722281 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2908d6-7c6b-40c9-aaf3-4088512812e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:25.722561 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2908d6-7c6b-40c9-aaf3-4088512812e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:25.722832 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2908d6-7c6b-40c9-aaf3-4088512812e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:25.723459 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2908d6-7c6b-40c9-aaf3-4088512812e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:25.725462 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2908d6-7c6b-40c9-aaf3-4088512812e7 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:25.760025 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1150/2302] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:44:25 2022] POST /identity/v3/projects => generated 363 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:44:25.765750 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8bd69d-0092-4ed1-9f6c-70e7cd0aef1f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:25.780012 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8bd69d-0092-4ed1-9f6c-70e7cd0aef1f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:25.839918 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87236b53-40bc-4a19-8001-31dbdee10f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:25.840540 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87236b53-40bc-4a19-8001-31dbdee10f26 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:25.840841 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87236b53-40bc-4a19-8001-31dbdee10f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:25.841118 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87236b53-40bc-4a19-8001-31dbdee10f26 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:25.841750 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87236b53-40bc-4a19-8001-31dbdee10f26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:25.843653 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87236b53-40bc-4a19-8001-31dbdee10f26 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:25.874426 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1153/2303] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:44:25 2022] POST /identity/v3/users => generated 282 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:44:25.874717 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2908d6-7c6b-40c9-aaf3-4088512812e7 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:25.887819 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2908d6-7c6b-40c9-aaf3-4088512812e7 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:25.892677 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9496ea0e-6ec3-4b4a-bf91-dd4b6b3832d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:25.893308 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9496ea0e-6ec3-4b4a-bf91-dd4b6b3832d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:25.893584 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9496ea0e-6ec3-4b4a-bf91-dd4b6b3832d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:25.893852 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9496ea0e-6ec3-4b4a-bf91-dd4b6b3832d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:25.894499 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9496ea0e-6ec3-4b4a-bf91-dd4b6b3832d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:25.896075 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9496ea0e-6ec3-4b4a-bf91-dd4b6b3832d2 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:25.908054 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1151/2304] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:44:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 09:44:25.911594 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87236b53-40bc-4a19-8001-31dbdee10f26 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:25.927367 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87236b53-40bc-4a19-8001-31dbdee10f26 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:25.934014 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e9b1f6-0779-44fc-b724-6c37adb89854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:25.934744 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e9b1f6-0779-44fc-b724-6c37adb89854 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:25.935031 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e9b1f6-0779-44fc-b724-6c37adb89854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:25.935304 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e9b1f6-0779-44fc-b724-6c37adb89854 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:25.935944 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e9b1f6-0779-44fc-b724-6c37adb89854 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:25.937604 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e9b1f6-0779-44fc-b724-6c37adb89854 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:25.953214 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1154/2305] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:44:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:44:25.954408 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9496ea0e-6ec3-4b4a-bf91-dd4b6b3832d2 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:25.966205 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9496ea0e-6ec3-4b4a-bf91-dd4b6b3832d2 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:25.972378 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41cb9b-31f6-4846-8168-596171e6e8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:25.973091 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41cb9b-31f6-4846-8168-596171e6e8a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:25.973368 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41cb9b-31f6-4846-8168-596171e6e8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:25.973638 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41cb9b-31f6-4846-8168-596171e6e8a8 admin admin] PATH_INFO: `/v3/projects/83d9b4c0e4a743188d2d0504c5e2fd11/users/aa44703169314dad96032f07e6b87547/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:25.991041 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41cb9b-31f6-4846-8168-596171e6e8a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83d9b4c0e4a743188d2d0504c5e2fd11, user_id=aa44703169314dad96032f07e6b87547, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:25.994402 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41cb9b-31f6-4846-8168-596171e6e8a8 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:26.010539 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1152/2306] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:44:25 2022] PUT /identity/v3/projects/83d9b4c0e4a743188d2d0504c5e2fd11/users/aa44703169314dad96032f07e6b87547/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:44:26.022500 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb24c4f0-4d0d-4c28-b8ce-97d08ac92f56 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:26.023059 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb24c4f0-4d0d-4c28-b8ce-97d08ac92f56 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:26.023174 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb24c4f0-4d0d-4c28-b8ce-97d08ac92f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:26.057889 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-cb24c4f0-4d0d-4c28-b8ce-97d08ac92f56 None None] MFA Rules not processed for user `aa44703169314dad96032f07e6b87547`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:26.059232 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-cb24c4f0-4d0d-4c28-b8ce-97d08ac92f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:26.134007 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1155/2307] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:44:26.139017 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41cb9b-31f6-4846-8168-596171e6e8a8 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:26.162648 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:26.210164 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41cb9b-31f6-4846-8168-596171e6e8a8 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:26.215913 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533f9234-57e7-41ac-a1c2-18ccce7be8a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:26.216570 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f9234-57e7-41ac-a1c2-18ccce7be8a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:26.216830 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f9234-57e7-41ac-a1c2-18ccce7be8a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:26.217103 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f9234-57e7-41ac-a1c2-18ccce7be8a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:26.217743 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533f9234-57e7-41ac-a1c2-18ccce7be8a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:26.219677 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533f9234-57e7-41ac-a1c2-18ccce7be8a7 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:26.253424 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1153/2308] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:44:26 2022] POST /identity/v3/projects => generated 365 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:44:26.255632 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb24c4f0-4d0d-4c28-b8ce-97d08ac92f56 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:26.267595 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb24c4f0-4d0d-4c28-b8ce-97d08ac92f56 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:26.315598 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:26.330662 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476cef25-799f-4201-b0c6-14f2301dd43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:26.331443 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476cef25-799f-4201-b0c6-14f2301dd43d admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:26.331817 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476cef25-799f-4201-b0c6-14f2301dd43d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:26.332775 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476cef25-799f-4201-b0c6-14f2301dd43d admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:26.333457 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476cef25-799f-4201-b0c6-14f2301dd43d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:26.336102 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476cef25-799f-4201-b0c6-14f2301dd43d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:26.363423 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1156/2309] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:44:26 2022] POST /identity/v3/users => generated 289 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:44:26.365029 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533f9234-57e7-41ac-a1c2-18ccce7be8a7 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:26.377490 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533f9234-57e7-41ac-a1c2-18ccce7be8a7 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:26.382405 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b66c25-3e2a-4fc2-9827-dcdf75753605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:26.383195 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b66c25-3e2a-4fc2-9827-dcdf75753605 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:26.383483 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b66c25-3e2a-4fc2-9827-dcdf75753605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:26.383742 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b66c25-3e2a-4fc2-9827-dcdf75753605 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:26.384383 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b66c25-3e2a-4fc2-9827-dcdf75753605 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:26.386089 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b66c25-3e2a-4fc2-9827-dcdf75753605 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:26.400952 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1154/2310] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:44:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:44:26.402210 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476cef25-799f-4201-b0c6-14f2301dd43d admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:26.412569 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476cef25-799f-4201-b0c6-14f2301dd43d admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:26.417783 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3ffd5b-b9ad-4fc3-8360-5e46e1d67463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:26.418455 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ffd5b-b9ad-4fc3-8360-5e46e1d67463 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:26.418728 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ffd5b-b9ad-4fc3-8360-5e46e1d67463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:26.418995 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ffd5b-b9ad-4fc3-8360-5e46e1d67463 admin admin] PATH_INFO: `/v3/projects/a3f9297c1fb449c0827083e1fbb53efa/users/a33b593f534c4915b443a153a1898d46/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:26.436032 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3ffd5b-b9ad-4fc3-8360-5e46e1d67463 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3f9297c1fb449c0827083e1fbb53efa, user_id=a33b593f534c4915b443a153a1898d46, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:26.438844 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3ffd5b-b9ad-4fc3-8360-5e46e1d67463 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:26.453961 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1157/2311] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:44:26 2022] PUT /identity/v3/projects/a3f9297c1fb449c0827083e1fbb53efa/users/a33b593f534c4915b443a153a1898d46/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:44:26.457946 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b66c25-3e2a-4fc2-9827-dcdf75753605 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:26.469278 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b66c25-3e2a-4fc2-9827-dcdf75753605 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:26.528234 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bea9715-bf40-4454-b972-4588bba8c3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:26.528883 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea9715-bf40-4454-b972-4588bba8c3b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:26.529158 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea9715-bf40-4454-b972-4588bba8c3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:26.529564 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea9715-bf40-4454-b972-4588bba8c3b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:26.530200 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bea9715-bf40-4454-b972-4588bba8c3b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:26.531807 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bea9715-bf40-4454-b972-4588bba8c3b9 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:26.542841 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1155/2312] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:44:26 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:44:26.547190 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3ffd5b-b9ad-4fc3-8360-5e46e1d67463 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:26.557780 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3ffd5b-b9ad-4fc3-8360-5e46e1d67463 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:26.562843 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64c4c71-0c8a-4bf8-893c-7801e764920e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:26.563523 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c4c71-0c8a-4bf8-893c-7801e764920e admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:26.563798 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c4c71-0c8a-4bf8-893c-7801e764920e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:26.564067 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c4c71-0c8a-4bf8-893c-7801e764920e admin admin] PATH_INFO: `/v3/projects/a3f9297c1fb449c0827083e1fbb53efa/users/a33b593f534c4915b443a153a1898d46/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:26.566167 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64c4c71-0c8a-4bf8-893c-7801e764920e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3f9297c1fb449c0827083e1fbb53efa, user_id=a33b593f534c4915b443a153a1898d46, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:26.568817 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64c4c71-0c8a-4bf8-893c-7801e764920e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:26.580638 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1158/2313] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:44:26 2022] PUT /identity/v3/projects/a3f9297c1fb449c0827083e1fbb53efa/users/a33b593f534c4915b443a153a1898d46/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:44:26.591660 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f5a31-842d-482d-8e34-cf1c63b474a3 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:26.591993 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f5a31-842d-482d-8e34-cf1c63b474a3 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:26.592204 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f5a31-842d-482d-8e34-cf1c63b474a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:26.621206 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a11f5a31-842d-482d-8e34-cf1c63b474a3 None None] MFA Rules not processed for user `a33b593f534c4915b443a153a1898d46`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:26.622420 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a11f5a31-842d-482d-8e34-cf1c63b474a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:26.696053 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1156/2314] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:26 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:44:26.708462 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64c4c71-0c8a-4bf8-893c-7801e764920e admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:26.709977 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b64c4c71-0c8a-4bf8-893c-7801e764920e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:26.787966 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64c4c71-0c8a-4bf8-893c-7801e764920e admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:26.794424 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859b1f51-7e62-48c7-a98c-2f1051b54641 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:26.795188 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859b1f51-7e62-48c7-a98c-2f1051b54641 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:26.796414 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859b1f51-7e62-48c7-a98c-2f1051b54641 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:26.796959 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859b1f51-7e62-48c7-a98c-2f1051b54641 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:26.819215 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859b1f51-7e62-48c7-a98c-2f1051b54641 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:26.823144 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859b1f51-7e62-48c7-a98c-2f1051b54641 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:26.847109 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1159/2315] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:44:26 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:44:26.936545 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11f5a31-842d-482d-8e34-cf1c63b474a3 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:26.946549 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11f5a31-842d-482d-8e34-cf1c63b474a3 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:26.950372 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea912f2-aea7-44ab-adec-3c3e7dff2a0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:26.950974 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea912f2-aea7-44ab-adec-3c3e7dff2a0e service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:26.951250 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea912f2-aea7-44ab-adec-3c3e7dff2a0e service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:26.951517 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea912f2-aea7-44ab-adec-3c3e7dff2a0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:26.964646 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:27.014524 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea912f2-aea7-44ab-adec-3c3e7dff2a0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:27.015967 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea912f2-aea7-44ab-adec-3c3e7dff2a0e service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:27.025815 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1157/2316] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:44:26 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:44:28.217823 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859b1f51-7e62-48c7-a98c-2f1051b54641 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:28.219253 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-859b1f51-7e62-48c7-a98c-2f1051b54641 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:28.314940 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859b1f51-7e62-48c7-a98c-2f1051b54641 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:28.321558 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c58cc8a-30b9-40c6-a86b-043183956c7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:28.322175 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c58cc8a-30b9-40c6-a86b-043183956c7a service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:28.322450 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c58cc8a-30b9-40c6-a86b-043183956c7a service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:28.322983 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c58cc8a-30b9-40c6-a86b-043183956c7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:28.325955 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2c58cc8a-30b9-40c6-a86b-043183956c7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:28.405096 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:28.409075 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c58cc8a-30b9-40c6-a86b-043183956c7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:28.411143 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c58cc8a-30b9-40c6-a86b-043183956c7a service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:28.424176 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1160/2317] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:44:28 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 09:44:35.085858 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea912f2-aea7-44ab-adec-3c3e7dff2a0e service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:35.087865 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6ea912f2-aea7-44ab-adec-3c3e7dff2a0e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:35.127954 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea912f2-aea7-44ab-adec-3c3e7dff2a0e service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:35.131583 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0533f2-e422-4f82-a93e-1631ee4fb5d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:35.132181 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0533f2-e422-4f82-a93e-1631ee4fb5d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:35.132458 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0533f2-e422-4f82-a93e-1631ee4fb5d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:35.132758 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0533f2-e422-4f82-a93e-1631ee4fb5d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:35.134982 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9b0533f2-e422-4f82-a93e-1631ee4fb5d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:35.248107 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0533f2-e422-4f82-a93e-1631ee4fb5d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:35.249595 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0533f2-e422-4f82-a93e-1631ee4fb5d5 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:35.270186 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1158/2318] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:44:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:44:36.738313 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c58cc8a-30b9-40c6-a86b-043183956c7a service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:36.778307 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c58cc8a-30b9-40c6-a86b-043183956c7a service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:36.783179 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192bb63-8271-4a4f-9fdd-cda65759ff1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:36.784061 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192bb63-8271-4a4f-9fdd-cda65759ff1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:36.784448 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192bb63-8271-4a4f-9fdd-cda65759ff1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:36.784887 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192bb63-8271-4a4f-9fdd-cda65759ff1d admin admin] PATH_INFO: `/v3/users/aa44703169314dad96032f07e6b87547` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:36.786140 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9192bb63-8271-4a4f-9fdd-cda65759ff1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa44703169314dad96032f07e6b87547)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:36.788159 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9192bb63-8271-4a4f-9fdd-cda65759ff1d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:36.844132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9192bb63-8271-4a4f-9fdd-cda65759ff1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa44703169314dad96032f07e6b87547'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:36.853770 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9192bb63-8271-4a4f-9fdd-cda65759ff1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa44703169314dad96032f07e6b87547'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:36.870510 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9192bb63-8271-4a4f-9fdd-cda65759ff1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa44703169314dad96032f07e6b87547'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:36.882511 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9192bb63-8271-4a4f-9fdd-cda65759ff1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa44703169314dad96032f07e6b87547'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:36.883810 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9192bb63-8271-4a4f-9fdd-cda65759ff1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa44703169314dad96032f07e6b87547'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:36.896902 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1161/2319] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:44:36 2022] DELETE /identity/v3/users/aa44703169314dad96032f07e6b87547 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:44:36.898651 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb9b62-0d17-4987-b736-4e779c027f21 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:36.899729 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb9b62-0d17-4987-b736-4e779c027f21 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:36.899849 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb9b62-0d17-4987-b736-4e779c027f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:36.931801 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-fbdb9b62-0d17-4987-b736-4e779c027f21 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:36.933195 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fbdb9b62-0d17-4987-b736-4e779c027f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:36.953455 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:37.003387 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1159/2320] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:44:37.014347 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192bb63-8271-4a4f-9fdd-cda65759ff1d admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:37.015784 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9192bb63-8271-4a4f-9fdd-cda65759ff1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:37.086235 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192bb63-8271-4a4f-9fdd-cda65759ff1d admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:37.091339 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bc82cb-aef1-4b31-bdcb-2394f47b39d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:37.091960 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc82cb-aef1-4b31-bdcb-2394f47b39d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:37.092239 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc82cb-aef1-4b31-bdcb-2394f47b39d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:37.092509 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc82cb-aef1-4b31-bdcb-2394f47b39d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:37.107034 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bc82cb-aef1-4b31-bdcb-2394f47b39d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:37.108712 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bc82cb-aef1-4b31-bdcb-2394f47b39d2 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:37.120792 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1162/2321] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:44:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:44:37.920194 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb9b62-0d17-4987-b736-4e779c027f21 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:37.922217 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fbdb9b62-0d17-4987-b736-4e779c027f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:37.958276 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb9b62-0d17-4987-b736-4e779c027f21 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:37.987502 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f1a8f2-5f1b-40a2-9401-c0e9242bce77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:37.991322 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f1a8f2-5f1b-40a2-9401-c0e9242bce77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:37.991681 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f1a8f2-5f1b-40a2-9401-c0e9242bce77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:37.992191 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f1a8f2-5f1b-40a2-9401-c0e9242bce77 admin admin] PATH_INFO: `/v3/projects/83d9b4c0e4a743188d2d0504c5e2fd11` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:37.993848 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f1a8f2-5f1b-40a2-9401-c0e9242bce77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83d9b4c0e4a743188d2d0504c5e2fd11)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:37.997772 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f1a8f2-5f1b-40a2-9401-c0e9242bce77 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:38.078565 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-36f1a8f2-5f1b-40a2-9401-c0e9242bce77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83d9b4c0e4a743188d2d0504c5e2fd11', 'request_id': 'req-36f1a8f2-5f1b-40a2-9401-c0e9242bce77'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:38.089170 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-36f1a8f2-5f1b-40a2-9401-c0e9242bce77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83d9b4c0e4a743188d2d0504c5e2fd11', 'request_id': 'req-36f1a8f2-5f1b-40a2-9401-c0e9242bce77'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:38.096079 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-36f1a8f2-5f1b-40a2-9401-c0e9242bce77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83d9b4c0e4a743188d2d0504c5e2fd11', 'request_id': 'req-36f1a8f2-5f1b-40a2-9401-c0e9242bce77'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:38.113707 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-36f1a8f2-5f1b-40a2-9401-c0e9242bce77 admin admin] The token cache is being invalidate because project 83d9b4c0e4a743188d2d0504c5e2fd11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:44:38.114018 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-36f1a8f2-5f1b-40a2-9401-c0e9242bce77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:38.114774 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-36f1a8f2-5f1b-40a2-9401-c0e9242bce77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:38.116394 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1160/2322] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:44:37 2022] DELETE /identity/v3/projects/83d9b4c0e4a743188d2d0504c5e2fd11 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:44:38.120357 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bc82cb-aef1-4b31-bdcb-2394f47b39d2 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:38.131031 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bc82cb-aef1-4b31-bdcb-2394f47b39d2 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:38.181098 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:38.202344 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6bd47-7e76-4191-80fb-f6c364ddf534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:38.203386 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6bd47-7e76-4191-80fb-f6c364ddf534 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:38.203738 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6bd47-7e76-4191-80fb-f6c364ddf534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:38.204079 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6bd47-7e76-4191-80fb-f6c364ddf534 admin admin] PATH_INFO: `/v3/users/a33b593f534c4915b443a153a1898d46` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:38.205424 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b6bd47-7e76-4191-80fb-f6c364ddf534 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a33b593f534c4915b443a153a1898d46)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:38.207447 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b6bd47-7e76-4191-80fb-f6c364ddf534 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:38.278366 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b1b6bd47-7e76-4191-80fb-f6c364ddf534 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a33b593f534c4915b443a153a1898d46'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:38.287687 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b1b6bd47-7e76-4191-80fb-f6c364ddf534 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a33b593f534c4915b443a153a1898d46'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:38.303945 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b1b6bd47-7e76-4191-80fb-f6c364ddf534 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a33b593f534c4915b443a153a1898d46'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:38.316212 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b1b6bd47-7e76-4191-80fb-f6c364ddf534 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a33b593f534c4915b443a153a1898d46'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:38.317479 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b1b6bd47-7e76-4191-80fb-f6c364ddf534 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a33b593f534c4915b443a153a1898d46'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:38.325870 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1163/2323] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:44:38 2022] DELETE /identity/v3/users/a33b593f534c4915b443a153a1898d46 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:44:38.770474 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f1a8f2-5f1b-40a2-9401-c0e9242bce77 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:38.772858 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-36f1a8f2-5f1b-40a2-9401-c0e9242bce77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:38.910408 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f1a8f2-5f1b-40a2-9401-c0e9242bce77 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:38.944696 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:38.991834 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703711d3-efc8-4586-9282-c56fa4d07ff4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2316b30a6e7b4175bd7b90583584eab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:38.993376 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703711d3-efc8-4586-9282-c56fa4d07ff4 service placement] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:38.993862 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703711d3-efc8-4586-9282-c56fa4d07ff4 service placement] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:38.994774 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703711d3-efc8-4586-9282-c56fa4d07ff4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:38.999446 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-703711d3-efc8-4586-9282-c56fa4d07ff4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:39.017413 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703711d3-efc8-4586-9282-c56fa4d07ff4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:39.019161 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703711d3-efc8-4586-9282-c56fa4d07ff4 service placement] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:39.029583 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1161/2324] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:44:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 09:44:39.051758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6bd47-7e76-4191-80fb-f6c364ddf534 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:39.053133 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b1b6bd47-7e76-4191-80fb-f6c364ddf534 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:39.206924 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6bd47-7e76-4191-80fb-f6c364ddf534 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:39.218806 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6535314-4322-42c4-b7eb-dabe482a227c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:39.219464 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6535314-4322-42c4-b7eb-dabe482a227c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:39.219750 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6535314-4322-42c4-b7eb-dabe482a227c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:39.220025 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6535314-4322-42c4-b7eb-dabe482a227c admin admin] PATH_INFO: `/v3/projects/a3f9297c1fb449c0827083e1fbb53efa` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:39.221269 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6535314-4322-42c4-b7eb-dabe482a227c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3f9297c1fb449c0827083e1fbb53efa)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:39.223157 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6535314-4322-42c4-b7eb-dabe482a227c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:39.287970 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f6535314-4322-42c4-b7eb-dabe482a227c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3f9297c1fb449c0827083e1fbb53efa', 'request_id': 'req-f6535314-4322-42c4-b7eb-dabe482a227c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:39.295779 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f6535314-4322-42c4-b7eb-dabe482a227c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3f9297c1fb449c0827083e1fbb53efa', 'request_id': 'req-f6535314-4322-42c4-b7eb-dabe482a227c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:39.306298 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f6535314-4322-42c4-b7eb-dabe482a227c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3f9297c1fb449c0827083e1fbb53efa', 'request_id': 'req-f6535314-4322-42c4-b7eb-dabe482a227c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:39.307983 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:39.321764 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f6535314-4322-42c4-b7eb-dabe482a227c admin admin] The token cache is being invalidate because project a3f9297c1fb449c0827083e1fbb53efa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:44:39.322056 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f6535314-4322-42c4-b7eb-dabe482a227c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:39.322975 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f6535314-4322-42c4-b7eb-dabe482a227c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:39.324489 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1164/2325] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:44:39 2022] DELETE /identity/v3/projects/a3f9297c1fb449c0827083e1fbb53efa => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:44:39.329452 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bb6713-9bde-4338-a8a3-a7963262e924 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:39.329739 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bb6713-9bde-4338-a8a3-a7963262e924 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:39.330012 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bb6713-9bde-4338-a8a3-a7963262e924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:39.385265 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-99bb6713-9bde-4338-a8a3-a7963262e924 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:39.386664 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-99bb6713-9bde-4338-a8a3-a7963262e924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:39.428686 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6535314-4322-42c4-b7eb-dabe482a227c admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:39.430191 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f6535314-4322-42c4-b7eb-dabe482a227c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:39.460345 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1162/2326] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:44:39.468985 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb0e06-7981-4262-9e59-1c32c09b7186 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:39.469275 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb0e06-7981-4262-9e59-1c32c09b7186 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:39.469550 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb0e06-7981-4262-9e59-1c32c09b7186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:39.489360 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-07cb0e06-7981-4262-9e59-1c32c09b7186 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:39.490608 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-07cb0e06-7981-4262-9e59-1c32c09b7186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:39.498468 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6535314-4322-42c4-b7eb-dabe482a227c admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:39.499025 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1163/2327] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:44:39.503203 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cb0e06-7981-4262-9e59-1c32c09b7186 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:39.503632 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f3bfa2-999a-4b79-9099-85ae0cd1db9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:39.504281 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3bfa2-999a-4b79-9099-85ae0cd1db9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:39.504563 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3bfa2-999a-4b79-9099-85ae0cd1db9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:39.504835 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3bfa2-999a-4b79-9099-85ae0cd1db9c admin admin] PATH_INFO: `/v3/users/f7d2c6275669404db608d714baeb0065` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:39.505949 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f3bfa2-999a-4b79-9099-85ae0cd1db9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7d2c6275669404db608d714baeb0065)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:39.507915 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f3bfa2-999a-4b79-9099-85ae0cd1db9c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:39.512818 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cb0e06-7981-4262-9e59-1c32c09b7186 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:39.517760 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda58ea-c8a1-4730-ae5c-93e6ed465e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:39.518376 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda58ea-c8a1-4730-ae5c-93e6ed465e0d admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:39.518686 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda58ea-c8a1-4730-ae5c-93e6ed465e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:39.519390 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda58ea-c8a1-4730-ae5c-93e6ed465e0d admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:39.520038 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adda58ea-c8a1-4730-ae5c-93e6ed465e0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:44:39.520341 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adda58ea-c8a1-4730-ae5c-93e6ed465e0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:39.522164 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adda58ea-c8a1-4730-ae5c-93e6ed465e0d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:39.538926 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1164/2328] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:44:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:44:39.540749 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda58ea-c8a1-4730-ae5c-93e6ed465e0d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:39.551089 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda58ea-c8a1-4730-ae5c-93e6ed465e0d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:39.555717 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbce7a08-cb56-4f96-9ab4-04f295df10c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:39.556298 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce7a08-cb56-4f96-9ab4-04f295df10c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:39.556601 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce7a08-cb56-4f96-9ab4-04f295df10c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:39.556868 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce7a08-cb56-4f96-9ab4-04f295df10c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:39.557426 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-64f3bfa2-999a-4b79-9099-85ae0cd1db9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7d2c6275669404db608d714baeb0065'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:39.557538 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbce7a08-cb56-4f96-9ab4-04f295df10c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:39.559346 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbce7a08-cb56-4f96-9ab4-04f295df10c2 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:39.564476 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-64f3bfa2-999a-4b79-9099-85ae0cd1db9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7d2c6275669404db608d714baeb0065'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:39.577410 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-64f3bfa2-999a-4b79-9099-85ae0cd1db9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7d2c6275669404db608d714baeb0065'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:39.587019 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-64f3bfa2-999a-4b79-9099-85ae0cd1db9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7d2c6275669404db608d714baeb0065'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:39.587875 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-64f3bfa2-999a-4b79-9099-85ae0cd1db9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7d2c6275669404db608d714baeb0065'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:39.594643 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1165/2329] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:44:39 2022] DELETE /identity/v3/users/f7d2c6275669404db608d714baeb0065 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:44:39.595071 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1165/2330] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:44:39 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:44:39.599172 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f3bfa2-999a-4b79-9099-85ae0cd1db9c admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:39.600612 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-64f3bfa2-999a-4b79-9099-85ae0cd1db9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:39.668805 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f3bfa2-999a-4b79-9099-85ae0cd1db9c admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:39.687003 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:39.734953 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22369246-7652-41f2-91b2-c2926d0acc20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:39.735720 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22369246-7652-41f2-91b2-c2926d0acc20 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:39.736092 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22369246-7652-41f2-91b2-c2926d0acc20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:39.736604 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22369246-7652-41f2-91b2-c2926d0acc20 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:39.737216 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22369246-7652-41f2-91b2-c2926d0acc20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:39.739449 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22369246-7652-41f2-91b2-c2926d0acc20 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:39.765606 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1166/2331] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:44:39 2022] POST /identity/v3/users => generated 289 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:44:39.768806 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbce7a08-cb56-4f96-9ab4-04f295df10c2 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:39.780955 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbce7a08-cb56-4f96-9ab4-04f295df10c2 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:39.788750 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22369246-7652-41f2-91b2-c2926d0acc20 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:39.790484 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-22369246-7652-41f2-91b2-c2926d0acc20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:39.791181 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353161b5-95d1-4b3c-9dbb-4895265b9109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:39.792097 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353161b5-95d1-4b3c-9dbb-4895265b9109 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:39.792579 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353161b5-95d1-4b3c-9dbb-4895265b9109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:39.793018 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353161b5-95d1-4b3c-9dbb-4895265b9109 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:39.793879 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353161b5-95d1-4b3c-9dbb-4895265b9109 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:39.795948 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353161b5-95d1-4b3c-9dbb-4895265b9109 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:39.804250 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22369246-7652-41f2-91b2-c2926d0acc20 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:39.807584 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1166/2332] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:44:39 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:44:39.809729 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cab923-5c71-4eb4-8b5c-4993f3d5fc9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:39.810355 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cab923-5c71-4eb4-8b5c-4993f3d5fc9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:39.810632 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cab923-5c71-4eb4-8b5c-4993f3d5fc9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:39.810902 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cab923-5c71-4eb4-8b5c-4993f3d5fc9c admin admin] PATH_INFO: `/v3/projects/df84eaa566a24d3e8d25c1065e8c09fa` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:39.812021 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cab923-5c71-4eb4-8b5c-4993f3d5fc9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=df84eaa566a24d3e8d25c1065e8c09fa)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:39.813954 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cab923-5c71-4eb4-8b5c-4993f3d5fc9c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:39.815644 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353161b5-95d1-4b3c-9dbb-4895265b9109 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:39.827175 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353161b5-95d1-4b3c-9dbb-4895265b9109 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:39.831875 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6550ff-f4cc-405d-9842-2de13f808f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:39.832469 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6550ff-f4cc-405d-9842-2de13f808f32 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:39.832777 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6550ff-f4cc-405d-9842-2de13f808f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:39.833042 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6550ff-f4cc-405d-9842-2de13f808f32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:39.833708 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6550ff-f4cc-405d-9842-2de13f808f32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:39.835306 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6550ff-f4cc-405d-9842-2de13f808f32 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:39.850347 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1167/2333] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:44:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:44:39.851279 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6550ff-f4cc-405d-9842-2de13f808f32 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:39.861022 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6550ff-f4cc-405d-9842-2de13f808f32 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:39.868465 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-01cab923-5c71-4eb4-8b5c-4993f3d5fc9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df84eaa566a24d3e8d25c1065e8c09fa', 'request_id': 'req-01cab923-5c71-4eb4-8b5c-4993f3d5fc9c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:39.874254 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-01cab923-5c71-4eb4-8b5c-4993f3d5fc9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df84eaa566a24d3e8d25c1065e8c09fa', 'request_id': 'req-01cab923-5c71-4eb4-8b5c-4993f3d5fc9c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:39.882085 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-01cab923-5c71-4eb4-8b5c-4993f3d5fc9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df84eaa566a24d3e8d25c1065e8c09fa', 'request_id': 'req-01cab923-5c71-4eb4-8b5c-4993f3d5fc9c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:39.891878 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:39.895185 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-01cab923-5c71-4eb4-8b5c-4993f3d5fc9c admin admin] The token cache is being invalidate because project df84eaa566a24d3e8d25c1065e8c09fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:44:39.895546 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-01cab923-5c71-4eb4-8b5c-4993f3d5fc9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:39.896300 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-01cab923-5c71-4eb4-8b5c-4993f3d5fc9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:39.898124 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1167/2334] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:44:39 2022] DELETE /identity/v3/projects/df84eaa566a24d3e8d25c1065e8c09fa => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:44:39.904824 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f47e88e-9a8b-4ac8-aa03-0f916d183c45 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:39.905107 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f47e88e-9a8b-4ac8-aa03-0f916d183c45 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:39.905381 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f47e88e-9a8b-4ac8-aa03-0f916d183c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:39.924827 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80136506-76d5-4d4b-8f6f-5d3430be56e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:39.925032 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-1f47e88e-9a8b-4ac8-aa03-0f916d183c45 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:39.925479 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80136506-76d5-4d4b-8f6f-5d3430be56e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:39.925756 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80136506-76d5-4d4b-8f6f-5d3430be56e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:39.926025 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80136506-76d5-4d4b-8f6f-5d3430be56e8 admin admin] PATH_INFO: `/v3/projects/c4417fe411d949ab9eeaa4366ac0be95/users/f1c5959d00a74e16b9ec23ed075a4731/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:39.926143 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1f47e88e-9a8b-4ac8-aa03-0f916d183c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:39.934608 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1168/2335] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:44:39.940631 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80136506-76d5-4d4b-8f6f-5d3430be56e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4417fe411d949ab9eeaa4366ac0be95, user_id=f1c5959d00a74e16b9ec23ed075a4731, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:39.942357 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd927ddf-77bb-4ba5-b1f2-bff4c488c46a None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:39.942643 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd927ddf-77bb-4ba5-b1f2-bff4c488c46a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:39.943040 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd927ddf-77bb-4ba5-b1f2-bff4c488c46a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:39.943270 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80136506-76d5-4d4b-8f6f-5d3430be56e8 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:39.959344 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1168/2336] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:44:39 2022] PUT /identity/v3/projects/c4417fe411d949ab9eeaa4366ac0be95/users/f1c5959d00a74e16b9ec23ed075a4731/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:44:39.971524 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584af4dc-1327-40f0-b03f-e2da3d8bb108 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:39.971889 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584af4dc-1327-40f0-b03f-e2da3d8bb108 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:39.972210 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584af4dc-1327-40f0-b03f-e2da3d8bb108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:39.975208 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-cd927ddf-77bb-4ba5-b1f2-bff4c488c46a None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:39.976425 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-cd927ddf-77bb-4ba5-b1f2-bff4c488c46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:40.005511 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-584af4dc-1327-40f0-b03f-e2da3d8bb108 None None] MFA Rules not processed for user `f1c5959d00a74e16b9ec23ed075a4731`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:40.007005 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-584af4dc-1327-40f0-b03f-e2da3d8bb108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:40.046646 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1169/2337] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:44:40.047998 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd927ddf-77bb-4ba5-b1f2-bff4c488c46a None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:40.060207 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd927ddf-77bb-4ba5-b1f2-bff4c488c46a None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:40.065369 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd01c0b-dced-4333-8707-b1e840f7c16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:40.066101 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd01c0b-dced-4333-8707-b1e840f7c16b admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:40.066380 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd01c0b-dced-4333-8707-b1e840f7c16b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:40.066648 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd01c0b-dced-4333-8707-b1e840f7c16b admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:40.067279 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd01c0b-dced-4333-8707-b1e840f7c16b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:44:40.067576 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd01c0b-dced-4333-8707-b1e840f7c16b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:40.069210 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd01c0b-dced-4333-8707-b1e840f7c16b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:40.081123 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1169/2338] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:44:40.083132 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1170/2339] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:44:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:44:40.085738 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584af4dc-1327-40f0-b03f-e2da3d8bb108 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:40.087623 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-584af4dc-1327-40f0-b03f-e2da3d8bb108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:40.088344 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd01c0b-dced-4333-8707-b1e840f7c16b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:40.098455 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd01c0b-dced-4333-8707-b1e840f7c16b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:40.104419 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6816cee2-4a93-4be6-89e8-b68f4b71e060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:40.104710 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584af4dc-1327-40f0-b03f-e2da3d8bb108 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:40.105153 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816cee2-4a93-4be6-89e8-b68f4b71e060 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:40.105301 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816cee2-4a93-4be6-89e8-b68f4b71e060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:40.105572 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816cee2-4a93-4be6-89e8-b68f4b71e060 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:40.106196 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6816cee2-4a93-4be6-89e8-b68f4b71e060 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:40.108136 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6816cee2-4a93-4be6-89e8-b68f4b71e060 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:40.110167 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b454a3-c910-4822-bf48-51318e48bf00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:40.110905 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b454a3-c910-4822-bf48-51318e48bf00 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:40.111255 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b454a3-c910-4822-bf48-51318e48bf00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:40.111647 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b454a3-c910-4822-bf48-51318e48bf00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:40.112347 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b454a3-c910-4822-bf48-51318e48bf00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:40.114348 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b454a3-c910-4822-bf48-51318e48bf00 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:40.120788 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:40.144551 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1171/2340] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:44:40 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:44:40.144775 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1170/2341] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:44:40 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:44:40.144883 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6816cee2-4a93-4be6-89e8-b68f4b71e060 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:40.146815 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b454a3-c910-4822-bf48-51318e48bf00 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:40.207830 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6816cee2-4a93-4be6-89e8-b68f4b71e060 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:40.209381 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b454a3-c910-4822-bf48-51318e48bf00 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:40.214890 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb76d320-6da9-4551-aa56-02a61fe47f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:40.215148 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7e7379-de30-4e63-b0bc-ebff6f1c4e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:40.215502 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb76d320-6da9-4551-aa56-02a61fe47f1c admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:40.215784 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb76d320-6da9-4551-aa56-02a61fe47f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:40.216056 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb76d320-6da9-4551-aa56-02a61fe47f1c admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:40.216653 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e7379-de30-4e63-b0bc-ebff6f1c4e98 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:40.217195 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb76d320-6da9-4551-aa56-02a61fe47f1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:40.217531 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e7379-de30-4e63-b0bc-ebff6f1c4e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:40.218228 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e7379-de30-4e63-b0bc-ebff6f1c4e98 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:40.218855 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7e7379-de30-4e63-b0bc-ebff6f1c4e98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:40.219251 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb76d320-6da9-4551-aa56-02a61fe47f1c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:40.221233 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7e7379-de30-4e63-b0bc-ebff6f1c4e98 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:40.244644 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1171/2342] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:44:40 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:44:40.247788 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1172/2343] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:44:40 2022] POST /identity/v3/users => generated 285 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:44:40.248375 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb76d320-6da9-4551-aa56-02a61fe47f1c admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:40.252310 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7e7379-de30-4e63-b0bc-ebff6f1c4e98 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:40.261678 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb76d320-6da9-4551-aa56-02a61fe47f1c admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:40.266949 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7e7379-de30-4e63-b0bc-ebff6f1c4e98 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:40.267744 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30231cfd-bf41-43a2-bdbc-b05ae69eaf09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:40.268462 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30231cfd-bf41-43a2-bdbc-b05ae69eaf09 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:40.268859 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30231cfd-bf41-43a2-bdbc-b05ae69eaf09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:40.269215 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30231cfd-bf41-43a2-bdbc-b05ae69eaf09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:40.270329 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30231cfd-bf41-43a2-bdbc-b05ae69eaf09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:40.272011 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30231cfd-bf41-43a2-bdbc-b05ae69eaf09 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:40.273123 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b01d99a-4106-45df-b97b-0b923c7dbf39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:40.274030 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01d99a-4106-45df-b97b-0b923c7dbf39 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:40.274311 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01d99a-4106-45df-b97b-0b923c7dbf39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:40.274830 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01d99a-4106-45df-b97b-0b923c7dbf39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:40.275483 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b01d99a-4106-45df-b97b-0b923c7dbf39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:40.277155 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b01d99a-4106-45df-b97b-0b923c7dbf39 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:40.287100 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1172/2344] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 09:44:40.287100 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbd73ef-5c22-4cb9-9b83-053db53377ed None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:40.287100 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbd73ef-5c22-4cb9-9b83-053db53377ed None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:40.287100 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbd73ef-5c22-4cb9-9b83-053db53377ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:40.288295 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1173/2345] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 09:44:40.289230 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b01d99a-4106-45df-b97b-0b923c7dbf39 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:40.294202 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:40.299887 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b01d99a-4106-45df-b97b-0b923c7dbf39 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:40.304561 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf2f7e-4993-470c-be6c-0be58a80bcef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:40.305772 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf2f7e-4993-470c-be6c-0be58a80bcef admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:40.306050 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf2f7e-4993-470c-be6c-0be58a80bcef admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:40.306324 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf2f7e-4993-470c-be6c-0be58a80bcef admin admin] PATH_INFO: `/v3/projects/e518ce5a5dfa458bad7206db1dc8290b/users/1fb10bf394d447b98b2c30a95e6c57d8/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:40.308446 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-fcbd73ef-5c22-4cb9-9b83-053db53377ed None None] MFA Rules not processed for user `52ac2b5469904e4a8a742de14fcc8508`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:40.309674 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fcbd73ef-5c22-4cb9-9b83-053db53377ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:40.322928 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf2f7e-4993-470c-be6c-0be58a80bcef admin admin] RBAC: Authorizing `identity:create_grant(project_id=e518ce5a5dfa458bad7206db1dc8290b, user_id=1fb10bf394d447b98b2c30a95e6c57d8, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:40.325630 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf2f7e-4993-470c-be6c-0be58a80bcef admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:40.339491 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1174/2346] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:44:40 2022] PUT /identity/v3/projects/e518ce5a5dfa458bad7206db1dc8290b/users/1fb10bf394d447b98b2c30a95e6c57d8/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:44:40.340323 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf2f7e-4993-470c-be6c-0be58a80bcef admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:40.366877 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1173/2347] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:44:40.368271 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbd73ef-5c22-4cb9-9b83-053db53377ed None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:40.378269 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbd73ef-5c22-4cb9-9b83-053db53377ed None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:40.416646 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf2f7e-4993-470c-be6c-0be58a80bcef admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:40.421661 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a429432-847a-4bd1-8f27-9cf82e7e29d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:40.422274 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a429432-847a-4bd1-8f27-9cf82e7e29d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:40.422553 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a429432-847a-4bd1-8f27-9cf82e7e29d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:40.422825 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a429432-847a-4bd1-8f27-9cf82e7e29d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:40.423862 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a429432-847a-4bd1-8f27-9cf82e7e29d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:40.425509 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a429432-847a-4bd1-8f27-9cf82e7e29d1 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:40.439478 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1175/2348] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:44:40.440605 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a429432-847a-4bd1-8f27-9cf82e7e29d1 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:40.441019 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d9da5-0d46-4d78-b170-50205d8636e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:40.441630 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d9da5-0d46-4d78-b170-50205d8636e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:40.441907 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d9da5-0d46-4d78-b170-50205d8636e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:40.442176 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d9da5-0d46-4d78-b170-50205d8636e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:40.442808 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8d9da5-0d46-4d78-b170-50205d8636e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:40.444408 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8d9da5-0d46-4d78-b170-50205d8636e1 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:40.451612 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a429432-847a-4bd1-8f27-9cf82e7e29d1 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:40.454907 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1174/2349] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:44:40.458587 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff1aeb9-91ba-45c2-892e-519f38fecc42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:40.459102 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d9da5-0d46-4d78-b170-50205d8636e1 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:40.459264 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1aeb9-91ba-45c2-892e-519f38fecc42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:40.459530 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1aeb9-91ba-45c2-892e-519f38fecc42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:40.459807 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1aeb9-91ba-45c2-892e-519f38fecc42 admin admin] PATH_INFO: `/v3/projects/93674d24fae245d0adebb9cf2727a5da/users/f425a3e552db4f61ba3ce4dc64c220bd/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:40.469605 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d9da5-0d46-4d78-b170-50205d8636e1 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:40.475386 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee60d3-b8a1-40f0-ae83-8419601c7096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:40.476041 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee60d3-b8a1-40f0-ae83-8419601c7096 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:40.476320 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee60d3-b8a1-40f0-ae83-8419601c7096 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:40.476655 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff1aeb9-91ba-45c2-892e-519f38fecc42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93674d24fae245d0adebb9cf2727a5da, user_id=f425a3e552db4f61ba3ce4dc64c220bd, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:40.477312 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee60d3-b8a1-40f0-ae83-8419601c7096 admin admin] PATH_INFO: `/v3/projects/e518ce5a5dfa458bad7206db1dc8290b/users/1fb10bf394d447b98b2c30a95e6c57d8/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:40.479667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acee60d3-b8a1-40f0-ae83-8419601c7096 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e518ce5a5dfa458bad7206db1dc8290b, user_id=1fb10bf394d447b98b2c30a95e6c57d8, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:40.480292 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff1aeb9-91ba-45c2-892e-519f38fecc42 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:40.482057 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acee60d3-b8a1-40f0-ae83-8419601c7096 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:40.496620 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1175/2350] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:44:40 2022] PUT /identity/v3/projects/e518ce5a5dfa458bad7206db1dc8290b/users/1fb10bf394d447b98b2c30a95e6c57d8/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:44:40.497972 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1176/2351] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:44:40 2022] PUT /identity/v3/projects/93674d24fae245d0adebb9cf2727a5da/users/f425a3e552db4f61ba3ce4dc64c220bd/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:44:40.504710 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9385f1-3f88-426e-9c8c-d7ff30087b9c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:40.504997 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9385f1-3f88-426e-9c8c-d7ff30087b9c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:40.505272 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9385f1-3f88-426e-9c8c-d7ff30087b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:40.508676 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de277e06-4ff8-47dd-a69e-54d3bddbbeb5 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:40.508967 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de277e06-4ff8-47dd-a69e-54d3bddbbeb5 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:40.509169 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de277e06-4ff8-47dd-a69e-54d3bddbbeb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:40.537906 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-3c9385f1-3f88-426e-9c8c-d7ff30087b9c None None] MFA Rules not processed for user `1fb10bf394d447b98b2c30a95e6c57d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:40.539127 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3c9385f1-3f88-426e-9c8c-d7ff30087b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:40.541358 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-de277e06-4ff8-47dd-a69e-54d3bddbbeb5 None None] MFA Rules not processed for user `f425a3e552db4f61ba3ce4dc64c220bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:40.542757 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-de277e06-4ff8-47dd-a69e-54d3bddbbeb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:40.548372 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:40.619585 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1176/2352] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:40 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:44:40.634190 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1177/2353] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:44:40.635181 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9385f1-3f88-426e-9c8c-d7ff30087b9c None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:40.636682 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de277e06-4ff8-47dd-a69e-54d3bddbbeb5 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:40.639108 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-de277e06-4ff8-47dd-a69e-54d3bddbbeb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:40.655661 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:40.705134 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de277e06-4ff8-47dd-a69e-54d3bddbbeb5 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:40.709173 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91305de-f04f-4e1e-b009-9647b2071376 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:40.709623 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9385f1-3f88-426e-9c8c-d7ff30087b9c None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:40.709779 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91305de-f04f-4e1e-b009-9647b2071376 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:40.710044 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91305de-f04f-4e1e-b009-9647b2071376 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:40.710314 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91305de-f04f-4e1e-b009-9647b2071376 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:40.715963 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9908b6-a9f7-412f-a932-81b41d69067c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:40.716709 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9908b6-a9f7-412f-a932-81b41d69067c admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:40.717059 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9908b6-a9f7-412f-a932-81b41d69067c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:40.717397 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9908b6-a9f7-412f-a932-81b41d69067c admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:40.718515 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9908b6-a9f7-412f-a932-81b41d69067c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:40.720445 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9908b6-a9f7-412f-a932-81b41d69067c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:40.754394 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1177/2354] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:44:40 2022] POST /identity/v3/projects => generated 367 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:44:40.755103 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9908b6-a9f7-412f-a932-81b41d69067c admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:40.777527 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91305de-f04f-4e1e-b009-9647b2071376 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:40.779206 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91305de-f04f-4e1e-b009-9647b2071376 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:40.790349 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1178/2355] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:44:40 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 09:44:40.831117 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9908b6-a9f7-412f-a932-81b41d69067c admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:40.837492 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa648dc-98b0-44f3-855a-b6230fc99d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:40.838250 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa648dc-98b0-44f3-855a-b6230fc99d50 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:40.838619 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa648dc-98b0-44f3-855a-b6230fc99d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:40.839109 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa648dc-98b0-44f3-855a-b6230fc99d50 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:40.840071 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa648dc-98b0-44f3-855a-b6230fc99d50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:40.842183 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa648dc-98b0-44f3-855a-b6230fc99d50 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:40.875847 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1178/2356] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:44:40 2022] POST /identity/v3/users => generated 290 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:44:40.876959 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91305de-f04f-4e1e-b009-9647b2071376 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:40.890654 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91305de-f04f-4e1e-b009-9647b2071376 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:40.896767 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c60478-c02f-4149-bff4-fb0d9edc81f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:40.897561 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c60478-c02f-4149-bff4-fb0d9edc81f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:40.897972 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c60478-c02f-4149-bff4-fb0d9edc81f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:40.899175 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c60478-c02f-4149-bff4-fb0d9edc81f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:40.899984 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c60478-c02f-4149-bff4-fb0d9edc81f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:40.901860 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c60478-c02f-4149-bff4-fb0d9edc81f0 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:40.913173 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1179/2357] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:44:40.918145 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa648dc-98b0-44f3-855a-b6230fc99d50 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:40.929405 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa648dc-98b0-44f3-855a-b6230fc99d50 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:40.935832 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dcf6c2-42d8-475e-a568-35aa2b151907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:40.936740 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dcf6c2-42d8-475e-a568-35aa2b151907 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:40.937127 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dcf6c2-42d8-475e-a568-35aa2b151907 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:40.937551 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dcf6c2-42d8-475e-a568-35aa2b151907 admin admin] PATH_INFO: `/v3/projects/d9dfc22bf9024f449ecf53159acf28f5/users/a3652d92c8b244d2ad08bc9f0348cc8d/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:40.956856 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dcf6c2-42d8-475e-a568-35aa2b151907 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9dfc22bf9024f449ecf53159acf28f5, user_id=a3652d92c8b244d2ad08bc9f0348cc8d, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:40.960746 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dcf6c2-42d8-475e-a568-35aa2b151907 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:40.995510 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1179/2358] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:44:40 2022] PUT /identity/v3/projects/d9dfc22bf9024f449ecf53159acf28f5/users/a3652d92c8b244d2ad08bc9f0348cc8d/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:44:40.999715 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c60478-c02f-4149-bff4-fb0d9edc81f0 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:41.013946 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c60478-c02f-4149-bff4-fb0d9edc81f0 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:41.115104 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:41.127816 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952401f-58a2-4b28-ac64-b09c5f0b8085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:41.128698 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952401f-58a2-4b28-ac64-b09c5f0b8085 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:41.129051 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952401f-58a2-4b28-ac64-b09c5f0b8085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:41.129403 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952401f-58a2-4b28-ac64-b09c5f0b8085 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:41.130637 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e952401f-58a2-4b28-ac64-b09c5f0b8085 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:41.133581 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e952401f-58a2-4b28-ac64-b09c5f0b8085 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:41.153930 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1180/2359] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:44:41.155270 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dcf6c2-42d8-475e-a568-35aa2b151907 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:41.169604 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dcf6c2-42d8-475e-a568-35aa2b151907 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:41.175403 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c266523-2104-47ac-a8a3-54bd0dbe1080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:41.175972 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c266523-2104-47ac-a8a3-54bd0dbe1080 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:41.176294 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c266523-2104-47ac-a8a3-54bd0dbe1080 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:41.176547 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c266523-2104-47ac-a8a3-54bd0dbe1080 admin admin] PATH_INFO: `/v3/projects/d9dfc22bf9024f449ecf53159acf28f5/users/a3652d92c8b244d2ad08bc9f0348cc8d/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:41.178663 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c266523-2104-47ac-a8a3-54bd0dbe1080 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9dfc22bf9024f449ecf53159acf28f5, user_id=a3652d92c8b244d2ad08bc9f0348cc8d, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:41.181394 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c266523-2104-47ac-a8a3-54bd0dbe1080 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:41.212568 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1180/2360] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:44:41 2022] PUT /identity/v3/projects/d9dfc22bf9024f449ecf53159acf28f5/users/a3652d92c8b244d2ad08bc9f0348cc8d/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:44:41.219543 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a4d9a0-5710-487f-add9-901dae6e4948 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:41.220036 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a4d9a0-5710-487f-add9-901dae6e4948 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:41.220299 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a4d9a0-5710-487f-add9-901dae6e4948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:41.284100 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-36a4d9a0-5710-487f-add9-901dae6e4948 None None] MFA Rules not processed for user `a3652d92c8b244d2ad08bc9f0348cc8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:41.287679 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-36a4d9a0-5710-487f-add9-901dae6e4948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:41.385496 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1181/2361] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:41 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:44:41.394857 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c266523-2104-47ac-a8a3-54bd0dbe1080 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:41.458334 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c266523-2104-47ac-a8a3-54bd0dbe1080 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:41.462897 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd757d28-37dd-4c21-bb80-2d5823830c8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:41.463616 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd757d28-37dd-4c21-bb80-2d5823830c8c service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:41.463971 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd757d28-37dd-4c21-bb80-2d5823830c8c service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:41.464308 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd757d28-37dd-4c21-bb80-2d5823830c8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:41.472915 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:41.475570 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd757d28-37dd-4c21-bb80-2d5823830c8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:41.476891 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd757d28-37dd-4c21-bb80-2d5823830c8c service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:41.489822 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1181/2362] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:44:41 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:44:43.008566 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a4d9a0-5710-487f-add9-901dae6e4948 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:43.068318 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a4d9a0-5710-487f-add9-901dae6e4948 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:43.073567 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6a04a3-a135-4bcc-bb85-3dc25de4b0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:43.074470 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6a04a3-a135-4bcc-bb85-3dc25de4b0c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:43.074860 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6a04a3-a135-4bcc-bb85-3dc25de4b0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:43.075255 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6a04a3-a135-4bcc-bb85-3dc25de4b0c1 admin admin] PATH_INFO: `/v3/users/f425a3e552db4f61ba3ce4dc64c220bd` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:43.076773 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6a04a3-a135-4bcc-bb85-3dc25de4b0c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f425a3e552db4f61ba3ce4dc64c220bd)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:43.079048 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6a04a3-a135-4bcc-bb85-3dc25de4b0c1 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:43.123780 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2f6a04a3-a135-4bcc-bb85-3dc25de4b0c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f425a3e552db4f61ba3ce4dc64c220bd'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:43.130515 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2f6a04a3-a135-4bcc-bb85-3dc25de4b0c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f425a3e552db4f61ba3ce4dc64c220bd'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:43.142035 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2f6a04a3-a135-4bcc-bb85-3dc25de4b0c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f425a3e552db4f61ba3ce4dc64c220bd'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:43.143631 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:43.150353 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2f6a04a3-a135-4bcc-bb85-3dc25de4b0c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f425a3e552db4f61ba3ce4dc64c220bd'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:43.151251 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2f6a04a3-a135-4bcc-bb85-3dc25de4b0c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f425a3e552db4f61ba3ce4dc64c220bd'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:43.156980 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1182/2363] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:44:43 2022] DELETE /identity/v3/users/f425a3e552db4f61ba3ce4dc64c220bd => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:44:43.161494 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e221f-d5af-4c9f-909b-14baaefc005b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:43.161962 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e221f-d5af-4c9f-909b-14baaefc005b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:43.162063 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e221f-d5af-4c9f-909b-14baaefc005b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:43.181048 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-6a0e221f-d5af-4c9f-909b-14baaefc005b None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:43.182232 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6a0e221f-d5af-4c9f-909b-14baaefc005b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:43.240740 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1182/2364] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:44:43.250166 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6a04a3-a135-4bcc-bb85-3dc25de4b0c1 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:43.251689 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2f6a04a3-a135-4bcc-bb85-3dc25de4b0c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:43.310380 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6a04a3-a135-4bcc-bb85-3dc25de4b0c1 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:43.314112 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d0d8a-3f8d-40df-b489-904f7fa58e64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:43.314723 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d0d8a-3f8d-40df-b489-904f7fa58e64 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:43.315003 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d0d8a-3f8d-40df-b489-904f7fa58e64 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:43.315271 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d0d8a-3f8d-40df-b489-904f7fa58e64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:43.325583 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77d0d8a-3f8d-40df-b489-904f7fa58e64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:43.327010 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77d0d8a-3f8d-40df-b489-904f7fa58e64 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:43.337375 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1183/2365] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:44:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:44:43.603435 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e221f-d5af-4c9f-909b-14baaefc005b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:43.605007 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6a0e221f-d5af-4c9f-909b-14baaefc005b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:43.618902 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e221f-d5af-4c9f-909b-14baaefc005b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:43.623539 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e96323d-dbd8-433b-a564-a08a7991c450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:43.624236 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e96323d-dbd8-433b-a564-a08a7991c450 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:43.625131 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e96323d-dbd8-433b-a564-a08a7991c450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:43.625637 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e96323d-dbd8-433b-a564-a08a7991c450 admin admin] PATH_INFO: `/v3/projects/93674d24fae245d0adebb9cf2727a5da` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:43.626978 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e96323d-dbd8-433b-a564-a08a7991c450 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93674d24fae245d0adebb9cf2727a5da)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:43.628825 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e96323d-dbd8-433b-a564-a08a7991c450 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:43.675460 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0e96323d-dbd8-433b-a564-a08a7991c450 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93674d24fae245d0adebb9cf2727a5da', 'request_id': 'req-0e96323d-dbd8-433b-a564-a08a7991c450'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:43.682060 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0e96323d-dbd8-433b-a564-a08a7991c450 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93674d24fae245d0adebb9cf2727a5da', 'request_id': 'req-0e96323d-dbd8-433b-a564-a08a7991c450'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:43.687629 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0e96323d-dbd8-433b-a564-a08a7991c450 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93674d24fae245d0adebb9cf2727a5da', 'request_id': 'req-0e96323d-dbd8-433b-a564-a08a7991c450'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:43.700777 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0e96323d-dbd8-433b-a564-a08a7991c450 admin admin] The token cache is being invalidate because project 93674d24fae245d0adebb9cf2727a5da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:44:43.701059 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0e96323d-dbd8-433b-a564-a08a7991c450 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:43.701664 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0e96323d-dbd8-433b-a564-a08a7991c450 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:43.703147 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1183/2366] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:44:43 2022] DELETE /identity/v3/projects/93674d24fae245d0adebb9cf2727a5da => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:44:43.707140 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d0d8a-3f8d-40df-b489-904f7fa58e64 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:43.716836 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d0d8a-3f8d-40df-b489-904f7fa58e64 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:43.771861 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebcd4bf-a0f2-47ae-8a04-5ff35c7f949b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:43.772475 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebcd4bf-a0f2-47ae-8a04-5ff35c7f949b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:43.772857 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebcd4bf-a0f2-47ae-8a04-5ff35c7f949b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:43.773131 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebcd4bf-a0f2-47ae-8a04-5ff35c7f949b admin admin] PATH_INFO: `/v3/users/a3652d92c8b244d2ad08bc9f0348cc8d` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:43.774237 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebcd4bf-a0f2-47ae-8a04-5ff35c7f949b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3652d92c8b244d2ad08bc9f0348cc8d)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:43.776128 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebcd4bf-a0f2-47ae-8a04-5ff35c7f949b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:43.820204 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8ebcd4bf-a0f2-47ae-8a04-5ff35c7f949b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3652d92c8b244d2ad08bc9f0348cc8d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:43.828066 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8ebcd4bf-a0f2-47ae-8a04-5ff35c7f949b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3652d92c8b244d2ad08bc9f0348cc8d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:43.839874 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8ebcd4bf-a0f2-47ae-8a04-5ff35c7f949b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3652d92c8b244d2ad08bc9f0348cc8d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:43.846260 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8ebcd4bf-a0f2-47ae-8a04-5ff35c7f949b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3652d92c8b244d2ad08bc9f0348cc8d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:43.846839 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8ebcd4bf-a0f2-47ae-8a04-5ff35c7f949b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3652d92c8b244d2ad08bc9f0348cc8d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:43.852603 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1184/2367] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:44:43 2022] DELETE /identity/v3/users/a3652d92c8b244d2ad08bc9f0348cc8d => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:44:44.119618 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e96323d-dbd8-433b-a564-a08a7991c450 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:44.121171 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0e96323d-dbd8-433b-a564-a08a7991c450 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:44.134555 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e96323d-dbd8-433b-a564-a08a7991c450 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:44.192046 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bf37b8-b891-4832-b8a2-3054a1845457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:44.192731 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bf37b8-b891-4832-b8a2-3054a1845457 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:44.192896 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bf37b8-b891-4832-b8a2-3054a1845457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:44.193167 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bf37b8-b891-4832-b8a2-3054a1845457 admin admin] PATH_INFO: `/v3/projects/d9dfc22bf9024f449ecf53159acf28f5` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:44.194277 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bf37b8-b891-4832-b8a2-3054a1845457 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9dfc22bf9024f449ecf53159acf28f5)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:44.196178 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bf37b8-b891-4832-b8a2-3054a1845457 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:44.218616 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:44.242505 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-64bf37b8-b891-4832-b8a2-3054a1845457 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9dfc22bf9024f449ecf53159acf28f5', 'request_id': 'req-64bf37b8-b891-4832-b8a2-3054a1845457'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:44.249023 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-64bf37b8-b891-4832-b8a2-3054a1845457 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9dfc22bf9024f449ecf53159acf28f5', 'request_id': 'req-64bf37b8-b891-4832-b8a2-3054a1845457'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:44.254752 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-64bf37b8-b891-4832-b8a2-3054a1845457 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9dfc22bf9024f449ecf53159acf28f5', 'request_id': 'req-64bf37b8-b891-4832-b8a2-3054a1845457'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:44.267553 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-64bf37b8-b891-4832-b8a2-3054a1845457 admin admin] The token cache is being invalidate because project d9dfc22bf9024f449ecf53159acf28f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:44:44.267924 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-64bf37b8-b891-4832-b8a2-3054a1845457 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:44.268642 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-64bf37b8-b891-4832-b8a2-3054a1845457 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:44.270173 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1184/2368] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:44:44 2022] DELETE /identity/v3/projects/d9dfc22bf9024f449ecf53159acf28f5 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:44:44.275155 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2436fe-7c26-4cad-9aa1-1a334d940b83 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:44.275613 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2436fe-7c26-4cad-9aa1-1a334d940b83 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:44.275732 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2436fe-7c26-4cad-9aa1-1a334d940b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:44.296103 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-0d2436fe-7c26-4cad-9aa1-1a334d940b83 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:44.297356 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0d2436fe-7c26-4cad-9aa1-1a334d940b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:44.305771 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:44.359396 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1185/2369] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:44:44.369470 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee383a-5b02-4737-a33c-e65811fe1789 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:44.369761 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee383a-5b02-4737-a33c-e65811fe1789 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:44.370035 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee383a-5b02-4737-a33c-e65811fe1789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:44.389707 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-afee383a-5b02-4737-a33c-e65811fe1789 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:44.390908 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-afee383a-5b02-4737-a33c-e65811fe1789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:44.398672 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1185/2370] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:44:44.403312 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2436fe-7c26-4cad-9aa1-1a334d940b83 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:44.413196 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2436fe-7c26-4cad-9aa1-1a334d940b83 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:44.418391 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad78f94d-e843-4365-ab53-f053e38e6738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:44.419039 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78f94d-e843-4365-ab53-f053e38e6738 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:44.419261 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78f94d-e843-4365-ab53-f053e38e6738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:44.419530 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78f94d-e843-4365-ab53-f053e38e6738 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:44.420215 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad78f94d-e843-4365-ab53-f053e38e6738 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:44:44.420482 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad78f94d-e843-4365-ab53-f053e38e6738 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:44.422096 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad78f94d-e843-4365-ab53-f053e38e6738 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:44.433805 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1186/2371] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:44:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:44:44.443355 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee383a-5b02-4737-a33c-e65811fe1789 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:44.453045 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee383a-5b02-4737-a33c-e65811fe1789 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:44.457622 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8888f145-afe7-4be0-a63c-3941b8dc2efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:44.458210 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8888f145-afe7-4be0-a63c-3941b8dc2efd admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:44.458481 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8888f145-afe7-4be0-a63c-3941b8dc2efd admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:44.458745 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8888f145-afe7-4be0-a63c-3941b8dc2efd admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:44.459373 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8888f145-afe7-4be0-a63c-3941b8dc2efd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:44.461324 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8888f145-afe7-4be0-a63c-3941b8dc2efd admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:44.489179 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1186/2372] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:44:44 2022] POST /identity/v3/projects => generated 367 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:44:44.493194 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad78f94d-e843-4365-ab53-f053e38e6738 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:44.504425 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad78f94d-e843-4365-ab53-f053e38e6738 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:44.564868 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74cebe7-a9ea-4d93-8d12-594c2cb92f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:44.565474 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74cebe7-a9ea-4d93-8d12-594c2cb92f61 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:44.565744 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74cebe7-a9ea-4d93-8d12-594c2cb92f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:44.566524 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74cebe7-a9ea-4d93-8d12-594c2cb92f61 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:44.566678 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74cebe7-a9ea-4d93-8d12-594c2cb92f61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:44.568596 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74cebe7-a9ea-4d93-8d12-594c2cb92f61 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:44.590614 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1187/2373] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:44:44 2022] POST /identity/v3/users => generated 284 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:44:44.597514 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8888f145-afe7-4be0-a63c-3941b8dc2efd admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:44.608415 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8888f145-afe7-4be0-a63c-3941b8dc2efd admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:44.613375 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7bfac4-cd51-4d38-b317-23a9eaa7b510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:44.613981 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7bfac4-cd51-4d38-b317-23a9eaa7b510 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:44.614260 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7bfac4-cd51-4d38-b317-23a9eaa7b510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:44.614527 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7bfac4-cd51-4d38-b317-23a9eaa7b510 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:44.615285 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7bfac4-cd51-4d38-b317-23a9eaa7b510 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:44.616915 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7bfac4-cd51-4d38-b317-23a9eaa7b510 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:44.627073 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1187/2374] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:44:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:44:44.630826 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74cebe7-a9ea-4d93-8d12-594c2cb92f61 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:44.640803 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74cebe7-a9ea-4d93-8d12-594c2cb92f61 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:44.645496 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c99a92-4168-43b1-b32f-8bd591fc9909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:44.646085 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c99a92-4168-43b1-b32f-8bd591fc9909 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:44.646363 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c99a92-4168-43b1-b32f-8bd591fc9909 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:44.646632 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c99a92-4168-43b1-b32f-8bd591fc9909 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:44.647260 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c99a92-4168-43b1-b32f-8bd591fc9909 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:44.648873 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c99a92-4168-43b1-b32f-8bd591fc9909 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:44.660071 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1188/2375] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:44:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:44:44.663974 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7bfac4-cd51-4d38-b317-23a9eaa7b510 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:44.674454 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7bfac4-cd51-4d38-b317-23a9eaa7b510 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:44.679097 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a8ee4-8f29-49f3-b4ee-1a9a6abae58d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:44.679730 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a8ee4-8f29-49f3-b4ee-1a9a6abae58d admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:44.680009 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a8ee4-8f29-49f3-b4ee-1a9a6abae58d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:44.680280 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a8ee4-8f29-49f3-b4ee-1a9a6abae58d admin admin] PATH_INFO: `/v3/projects/62193ce4c64645f5aef4d66278d9c67e/users/d8d06d57d2464e459d4f23313c58469a/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:44.694623 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045a8ee4-8f29-49f3-b4ee-1a9a6abae58d admin admin] RBAC: Authorizing `identity:create_grant(project_id=62193ce4c64645f5aef4d66278d9c67e, user_id=d8d06d57d2464e459d4f23313c58469a, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:44.697547 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045a8ee4-8f29-49f3-b4ee-1a9a6abae58d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:44.710314 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1188/2376] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:44:44 2022] PUT /identity/v3/projects/62193ce4c64645f5aef4d66278d9c67e/users/d8d06d57d2464e459d4f23313c58469a/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:44:44.721127 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc0fea4-32cd-45c2-9854-ab90428f46db None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:44.721453 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc0fea4-32cd-45c2-9854-ab90428f46db None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:44.721723 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc0fea4-32cd-45c2-9854-ab90428f46db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:44.749432 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-9cc0fea4-32cd-45c2-9854-ab90428f46db None None] MFA Rules not processed for user `d8d06d57d2464e459d4f23313c58469a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:44.750684 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9cc0fea4-32cd-45c2-9854-ab90428f46db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:44.813348 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1189/2377] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:44:44.817742 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a8ee4-8f29-49f3-b4ee-1a9a6abae58d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:44.879101 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a8ee4-8f29-49f3-b4ee-1a9a6abae58d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:44.883724 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83effc-b9a6-4568-a94c-87c8b45f2304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:44.884834 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83effc-b9a6-4568-a94c-87c8b45f2304 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:44.885115 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83effc-b9a6-4568-a94c-87c8b45f2304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:44.885387 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83effc-b9a6-4568-a94c-87c8b45f2304 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:44.885995 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e83effc-b9a6-4568-a94c-87c8b45f2304 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:44.887897 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e83effc-b9a6-4568-a94c-87c8b45f2304 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:44.914319 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1189/2378] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:44:44 2022] POST /identity/v3/projects => generated 365 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:44:44.918681 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc0fea4-32cd-45c2-9854-ab90428f46db None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:44.929086 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc0fea4-32cd-45c2-9854-ab90428f46db None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:44.933579 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:44.984863 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1dcdf-a622-4f40-9e7b-8a1f161ef47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:44.985385 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1dcdf-a622-4f40-9e7b-8a1f161ef47a admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:44.985659 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1dcdf-a622-4f40-9e7b-8a1f161ef47a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:44.985923 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1dcdf-a622-4f40-9e7b-8a1f161ef47a admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:44.986527 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf1dcdf-a622-4f40-9e7b-8a1f161ef47a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:44.988405 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf1dcdf-a622-4f40-9e7b-8a1f161ef47a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:45.008797 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1190/2379] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:44:44 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:44:45.013234 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83effc-b9a6-4568-a94c-87c8b45f2304 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:45.023549 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83effc-b9a6-4568-a94c-87c8b45f2304 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:45.028885 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9861bb5-66e4-4930-84f5-b1e17fb668d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:45.029474 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9861bb5-66e4-4930-84f5-b1e17fb668d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:45.029750 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9861bb5-66e4-4930-84f5-b1e17fb668d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:45.030018 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9861bb5-66e4-4930-84f5-b1e17fb668d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:45.030667 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9861bb5-66e4-4930-84f5-b1e17fb668d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:45.032318 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9861bb5-66e4-4930-84f5-b1e17fb668d3 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:45.042730 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1190/2380] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:44:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:44:45.046647 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1dcdf-a622-4f40-9e7b-8a1f161ef47a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:45.056470 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1dcdf-a622-4f40-9e7b-8a1f161ef47a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:45.060990 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc47221-4f20-4ab4-9498-f178dffd2a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:45.061623 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc47221-4f20-4ab4-9498-f178dffd2a16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:45.061895 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc47221-4f20-4ab4-9498-f178dffd2a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:45.062159 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc47221-4f20-4ab4-9498-f178dffd2a16 admin admin] PATH_INFO: `/v3/projects/be9956d474af49b89528cc6a79428df2/users/b605b85a40804eb69464ed754cd4d1ec/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:45.078998 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc47221-4f20-4ab4-9498-f178dffd2a16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be9956d474af49b89528cc6a79428df2, user_id=b605b85a40804eb69464ed754cd4d1ec, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:45.081869 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc47221-4f20-4ab4-9498-f178dffd2a16 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:45.094696 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1191/2381] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:44:45 2022] PUT /identity/v3/projects/be9956d474af49b89528cc6a79428df2/users/b605b85a40804eb69464ed754cd4d1ec/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:44:45.098747 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9861bb5-66e4-4930-84f5-b1e17fb668d3 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:45.108716 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9861bb5-66e4-4930-84f5-b1e17fb668d3 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:45.124584 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:45.164912 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94440529-008b-4cab-ba47-a86b62404f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:45.165528 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94440529-008b-4cab-ba47-a86b62404f4a admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:45.165812 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94440529-008b-4cab-ba47-a86b62404f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:45.166080 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94440529-008b-4cab-ba47-a86b62404f4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:45.166703 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94440529-008b-4cab-ba47-a86b62404f4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:45.168290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94440529-008b-4cab-ba47-a86b62404f4a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:45.177797 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1191/2382] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:44:45 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:44:45.182524 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc47221-4f20-4ab4-9498-f178dffd2a16 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:45.192742 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc47221-4f20-4ab4-9498-f178dffd2a16 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:45.198238 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32db740-521e-4962-855f-3a1ef67d99c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:45.198884 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32db740-521e-4962-855f-3a1ef67d99c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:45.199162 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32db740-521e-4962-855f-3a1ef67d99c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:45.199437 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32db740-521e-4962-855f-3a1ef67d99c8 admin admin] PATH_INFO: `/v3/projects/be9956d474af49b89528cc6a79428df2/users/b605b85a40804eb69464ed754cd4d1ec/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:45.201380 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32db740-521e-4962-855f-3a1ef67d99c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be9956d474af49b89528cc6a79428df2, user_id=b605b85a40804eb69464ed754cd4d1ec, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:45.204006 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32db740-521e-4962-855f-3a1ef67d99c8 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:45.214130 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1192/2383] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:44:45 2022] PUT /identity/v3/projects/be9956d474af49b89528cc6a79428df2/users/b605b85a40804eb69464ed754cd4d1ec/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:44:45.224200 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951ec04-af3b-45c5-a0cc-9ddbfd282825 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:45.224693 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951ec04-af3b-45c5-a0cc-9ddbfd282825 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:45.224818 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951ec04-af3b-45c5-a0cc-9ddbfd282825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:45.252705 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-5951ec04-af3b-45c5-a0cc-9ddbfd282825 None None] MFA Rules not processed for user `b605b85a40804eb69464ed754cd4d1ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:45.253839 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5951ec04-af3b-45c5-a0cc-9ddbfd282825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:45.320935 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1192/2384] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:45 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:44:45.333021 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32db740-521e-4962-855f-3a1ef67d99c8 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:45.334794 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f32db740-521e-4962-855f-3a1ef67d99c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:45.398450 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32db740-521e-4962-855f-3a1ef67d99c8 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:45.402175 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a31302-e202-464f-86be-7e68bdbbd9f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:45.402785 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a31302-e202-464f-86be-7e68bdbbd9f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:45.403063 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a31302-e202-464f-86be-7e68bdbbd9f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:45.403333 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a31302-e202-464f-86be-7e68bdbbd9f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:45.413206 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a31302-e202-464f-86be-7e68bdbbd9f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:45.414640 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a31302-e202-464f-86be-7e68bdbbd9f4 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:45.425488 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1193/2385] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:44:45 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:44:45.515526 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5951ec04-af3b-45c5-a0cc-9ddbfd282825 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:45.525764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5951ec04-af3b-45c5-a0cc-9ddbfd282825 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:45.529532 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b49e207-801b-48db-a131-003c77047487 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:45.530208 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49e207-801b-48db-a131-003c77047487 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:45.530556 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49e207-801b-48db-a131-003c77047487 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:45.530837 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49e207-801b-48db-a131-003c77047487 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:45.588408 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b49e207-801b-48db-a131-003c77047487 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:45.590269 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b49e207-801b-48db-a131-003c77047487 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:45.599869 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1193/2386] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:44:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:44:45.674531 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a31302-e202-464f-86be-7e68bdbbd9f4 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:45.676187 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f8a31302-e202-464f-86be-7e68bdbbd9f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:45.691099 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a31302-e202-464f-86be-7e68bdbbd9f4 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:45.699383 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a178c0dd-08cf-422d-8940-dff093c2b57a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:45.701803 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178c0dd-08cf-422d-8940-dff093c2b57a service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:45.703407 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178c0dd-08cf-422d-8940-dff093c2b57a service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:45.703407 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178c0dd-08cf-422d-8940-dff093c2b57a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:45.704398 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a178c0dd-08cf-422d-8940-dff093c2b57a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:45.789459 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a178c0dd-08cf-422d-8940-dff093c2b57a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:45.791661 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a178c0dd-08cf-422d-8940-dff093c2b57a service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:45.796302 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:45.811889 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1194/2387] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:44:45 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:44:50.397092 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b49e207-801b-48db-a131-003c77047487 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:50.439610 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:50.460428 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b49e207-801b-48db-a131-003c77047487 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:50.465468 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ceb8f0-8229-4764-9d9f-9b9e64abd058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:50.466089 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ceb8f0-8229-4764-9d9f-9b9e64abd058 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:50.466368 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ceb8f0-8229-4764-9d9f-9b9e64abd058 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:50.466639 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ceb8f0-8229-4764-9d9f-9b9e64abd058 admin admin] PATH_INFO: `/v3/users/d8d06d57d2464e459d4f23313c58469a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:50.467738 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ceb8f0-8229-4764-9d9f-9b9e64abd058 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8d06d57d2464e459d4f23313c58469a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:50.469703 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ceb8f0-8229-4764-9d9f-9b9e64abd058 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:50.515456 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-20ceb8f0-8229-4764-9d9f-9b9e64abd058 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8d06d57d2464e459d4f23313c58469a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:50.522739 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-20ceb8f0-8229-4764-9d9f-9b9e64abd058 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8d06d57d2464e459d4f23313c58469a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:50.536217 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-20ceb8f0-8229-4764-9d9f-9b9e64abd058 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8d06d57d2464e459d4f23313c58469a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:50.544541 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-20ceb8f0-8229-4764-9d9f-9b9e64abd058 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8d06d57d2464e459d4f23313c58469a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:50.545248 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-20ceb8f0-8229-4764-9d9f-9b9e64abd058 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8d06d57d2464e459d4f23313c58469a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:50.551221 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1194/2388] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:44:50 2022] DELETE /identity/v3/users/d8d06d57d2464e459d4f23313c58469a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:44:50.555847 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c5add-9294-4ae5-b5ef-c258f659c0ea None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:50.556355 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c5add-9294-4ae5-b5ef-c258f659c0ea None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:50.556548 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c5add-9294-4ae5-b5ef-c258f659c0ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:50.576856 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f28c5add-9294-4ae5-b5ef-c258f659c0ea None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:50.578062 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f28c5add-9294-4ae5-b5ef-c258f659c0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:50.646788 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1195/2389] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:44:50.654316 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ceb8f0-8229-4764-9d9f-9b9e64abd058 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:50.656413 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-20ceb8f0-8229-4764-9d9f-9b9e64abd058 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:50.740702 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ceb8f0-8229-4764-9d9f-9b9e64abd058 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:50.744463 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0450ad-113f-40b1-b63d-70789675c35c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:50.745100 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0450ad-113f-40b1-b63d-70789675c35c service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:50.745375 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0450ad-113f-40b1-b63d-70789675c35c service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:50.745645 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0450ad-113f-40b1-b63d-70789675c35c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:50.755550 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0450ad-113f-40b1-b63d-70789675c35c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:50.757032 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0450ad-113f-40b1-b63d-70789675c35c service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:50.767326 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1195/2390] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:44:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 09:44:51.072921 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c5add-9294-4ae5-b5ef-c258f659c0ea None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:51.074848 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f28c5add-9294-4ae5-b5ef-c258f659c0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:51.089240 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c5add-9294-4ae5-b5ef-c258f659c0ea None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:51.094036 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f17367-86ed-451d-80bb-912ca4c2ead8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:51.094702 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f17367-86ed-451d-80bb-912ca4c2ead8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:51.094982 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f17367-86ed-451d-80bb-912ca4c2ead8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:51.095251 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f17367-86ed-451d-80bb-912ca4c2ead8 admin admin] PATH_INFO: `/v3/projects/62193ce4c64645f5aef4d66278d9c67e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:51.096378 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f17367-86ed-451d-80bb-912ca4c2ead8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62193ce4c64645f5aef4d66278d9c67e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:51.098413 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f17367-86ed-451d-80bb-912ca4c2ead8 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:51.144905 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f8f17367-86ed-451d-80bb-912ca4c2ead8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62193ce4c64645f5aef4d66278d9c67e', 'request_id': 'req-f8f17367-86ed-451d-80bb-912ca4c2ead8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:51.151146 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f8f17367-86ed-451d-80bb-912ca4c2ead8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62193ce4c64645f5aef4d66278d9c67e', 'request_id': 'req-f8f17367-86ed-451d-80bb-912ca4c2ead8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:51.158118 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f8f17367-86ed-451d-80bb-912ca4c2ead8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62193ce4c64645f5aef4d66278d9c67e', 'request_id': 'req-f8f17367-86ed-451d-80bb-912ca4c2ead8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:51.169292 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f8f17367-86ed-451d-80bb-912ca4c2ead8 admin admin] The token cache is being invalidate because project 62193ce4c64645f5aef4d66278d9c67e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:44:51.169575 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f8f17367-86ed-451d-80bb-912ca4c2ead8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:51.170214 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f8f17367-86ed-451d-80bb-912ca4c2ead8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:51.171719 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1196/2391] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:44:51 2022] DELETE /identity/v3/projects/62193ce4c64645f5aef4d66278d9c67e => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:44:51.175722 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0450ad-113f-40b1-b63d-70789675c35c service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:51.185935 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0450ad-113f-40b1-b63d-70789675c35c service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:51.226107 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:51.244052 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec31c8e-6766-4c41-a2ba-3e5b881d121b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:51.244757 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec31c8e-6766-4c41-a2ba-3e5b881d121b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:51.245039 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec31c8e-6766-4c41-a2ba-3e5b881d121b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:51.245308 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec31c8e-6766-4c41-a2ba-3e5b881d121b admin admin] PATH_INFO: `/v3/users/b605b85a40804eb69464ed754cd4d1ec` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:51.246465 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec31c8e-6766-4c41-a2ba-3e5b881d121b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b605b85a40804eb69464ed754cd4d1ec)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:51.248378 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec31c8e-6766-4c41-a2ba-3e5b881d121b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:51.294023 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-dec31c8e-6766-4c41-a2ba-3e5b881d121b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b605b85a40804eb69464ed754cd4d1ec'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:51.301038 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-dec31c8e-6766-4c41-a2ba-3e5b881d121b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b605b85a40804eb69464ed754cd4d1ec'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:51.312826 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-dec31c8e-6766-4c41-a2ba-3e5b881d121b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b605b85a40804eb69464ed754cd4d1ec'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:51.320008 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-dec31c8e-6766-4c41-a2ba-3e5b881d121b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b605b85a40804eb69464ed754cd4d1ec'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:51.320659 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-dec31c8e-6766-4c41-a2ba-3e5b881d121b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b605b85a40804eb69464ed754cd4d1ec'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:51.326915 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1196/2392] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:44:51 2022] DELETE /identity/v3/users/b605b85a40804eb69464ed754cd4d1ec => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:44:51.594835 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f17367-86ed-451d-80bb-912ca4c2ead8 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:51.596352 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f8f17367-86ed-451d-80bb-912ca4c2ead8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:51.609564 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f17367-86ed-451d-80bb-912ca4c2ead8 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:51.667806 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcc248a-1e1b-46f8-9c1a-6ecf7f35f841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:51.668433 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc248a-1e1b-46f8-9c1a-6ecf7f35f841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:51.668748 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc248a-1e1b-46f8-9c1a-6ecf7f35f841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:51.669015 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc248a-1e1b-46f8-9c1a-6ecf7f35f841 admin admin] PATH_INFO: `/v3/projects/be9956d474af49b89528cc6a79428df2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:51.670117 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcc248a-1e1b-46f8-9c1a-6ecf7f35f841 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be9956d474af49b89528cc6a79428df2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:51.672002 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcc248a-1e1b-46f8-9c1a-6ecf7f35f841 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:51.695391 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:51.720504 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4dcc248a-1e1b-46f8-9c1a-6ecf7f35f841 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be9956d474af49b89528cc6a79428df2', 'request_id': 'req-4dcc248a-1e1b-46f8-9c1a-6ecf7f35f841'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:51.725852 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4dcc248a-1e1b-46f8-9c1a-6ecf7f35f841 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be9956d474af49b89528cc6a79428df2', 'request_id': 'req-4dcc248a-1e1b-46f8-9c1a-6ecf7f35f841'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:51.732894 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4dcc248a-1e1b-46f8-9c1a-6ecf7f35f841 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be9956d474af49b89528cc6a79428df2', 'request_id': 'req-4dcc248a-1e1b-46f8-9c1a-6ecf7f35f841'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:51.743560 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4dcc248a-1e1b-46f8-9c1a-6ecf7f35f841 admin admin] The token cache is being invalidate because project be9956d474af49b89528cc6a79428df2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:44:51.743842 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4dcc248a-1e1b-46f8-9c1a-6ecf7f35f841 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:51.744453 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4dcc248a-1e1b-46f8-9c1a-6ecf7f35f841 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:51.745939 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1197/2393] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:44:51 2022] DELETE /identity/v3/projects/be9956d474af49b89528cc6a79428df2 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:44:51.750756 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b678357-070e-4146-a8f5-c347305797ca None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:51.751218 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b678357-070e-4146-a8f5-c347305797ca None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:51.751316 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b678357-070e-4146-a8f5-c347305797ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:51.772033 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-7b678357-070e-4146-a8f5-c347305797ca None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:51.773260 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7b678357-070e-4146-a8f5-c347305797ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:51.832884 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1197/2394] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:44:51.841178 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dcea72-f5b3-455b-973e-e376cee9c431 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:51.841460 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dcea72-f5b3-455b-973e-e376cee9c431 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:51.841727 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dcea72-f5b3-455b-973e-e376cee9c431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:51.861179 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-99dcea72-f5b3-455b-973e-e376cee9c431 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:51.862367 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-99dcea72-f5b3-455b-973e-e376cee9c431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:51.870112 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1198/2395] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:44:51.874228 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b678357-070e-4146-a8f5-c347305797ca None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:51.883714 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b678357-070e-4146-a8f5-c347305797ca None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:51.888777 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0227fa39-420e-4fb6-84de-a93584b02380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:51.889364 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0227fa39-420e-4fb6-84de-a93584b02380 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:51.889638 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0227fa39-420e-4fb6-84de-a93584b02380 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:51.889903 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0227fa39-420e-4fb6-84de-a93584b02380 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:51.890537 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0227fa39-420e-4fb6-84de-a93584b02380 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:44:51.890834 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0227fa39-420e-4fb6-84de-a93584b02380 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:51.892413 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0227fa39-420e-4fb6-84de-a93584b02380 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:51.904265 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1198/2396] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:44:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:44:51.908236 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dcea72-f5b3-455b-973e-e376cee9c431 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:51.917328 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dcea72-f5b3-455b-973e-e376cee9c431 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:51.921861 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d8b0d7-8ebe-4369-9273-c77013cc1831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:51.922439 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8b0d7-8ebe-4369-9273-c77013cc1831 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:51.922711 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8b0d7-8ebe-4369-9273-c77013cc1831 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:51.922975 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8b0d7-8ebe-4369-9273-c77013cc1831 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:51.923598 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d8b0d7-8ebe-4369-9273-c77013cc1831 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:51.925501 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d8b0d7-8ebe-4369-9273-c77013cc1831 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:51.952828 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1199/2397] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:44:51 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:44:51.956673 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0227fa39-420e-4fb6-84de-a93584b02380 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:51.966994 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0227fa39-420e-4fb6-84de-a93584b02380 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:52.010907 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:52.024092 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96802f-6677-4c3b-bdc2-4ebbc8d30c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:52.024744 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96802f-6677-4c3b-bdc2-4ebbc8d30c77 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:52.025020 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96802f-6677-4c3b-bdc2-4ebbc8d30c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:52.025288 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96802f-6677-4c3b-bdc2-4ebbc8d30c77 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:52.025913 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab96802f-6677-4c3b-bdc2-4ebbc8d30c77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:52.027804 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab96802f-6677-4c3b-bdc2-4ebbc8d30c77 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:52.048250 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1199/2398] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:44:51 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:44:52.052405 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d8b0d7-8ebe-4369-9273-c77013cc1831 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:52.063437 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d8b0d7-8ebe-4369-9273-c77013cc1831 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:52.067984 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43114a0f-99b3-4708-a37c-0a607edc437b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:52.068605 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43114a0f-99b3-4708-a37c-0a607edc437b admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:52.068881 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43114a0f-99b3-4708-a37c-0a607edc437b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:52.069147 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43114a0f-99b3-4708-a37c-0a607edc437b admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:52.069768 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43114a0f-99b3-4708-a37c-0a607edc437b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:52.071326 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43114a0f-99b3-4708-a37c-0a607edc437b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:52.083597 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1200/2399] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:44:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:44:52.087707 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96802f-6677-4c3b-bdc2-4ebbc8d30c77 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:52.097904 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96802f-6677-4c3b-bdc2-4ebbc8d30c77 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:52.102446 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c4a85a-749a-47a8-a69a-a41745a70659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:52.103097 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c4a85a-749a-47a8-a69a-a41745a70659 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:52.103373 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c4a85a-749a-47a8-a69a-a41745a70659 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:52.103641 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c4a85a-749a-47a8-a69a-a41745a70659 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:52.104270 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c4a85a-749a-47a8-a69a-a41745a70659 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:52.105933 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c4a85a-749a-47a8-a69a-a41745a70659 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:52.116834 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1200/2400] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:44:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 09:44:52.121606 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43114a0f-99b3-4708-a37c-0a607edc437b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:52.131800 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43114a0f-99b3-4708-a37c-0a607edc437b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:52.137480 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0635e5b9-71b2-4e23-9626-623409a24de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:52.138209 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635e5b9-71b2-4e23-9626-623409a24de5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:52.138556 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635e5b9-71b2-4e23-9626-623409a24de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:52.138895 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635e5b9-71b2-4e23-9626-623409a24de5 admin admin] PATH_INFO: `/v3/projects/ed2fc67c801448d0a68ebb8f56a1bfb6/users/d5ed29d78c5f4ac592995dc802258f19/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:52.154521 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0635e5b9-71b2-4e23-9626-623409a24de5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed2fc67c801448d0a68ebb8f56a1bfb6, user_id=d5ed29d78c5f4ac592995dc802258f19, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:52.157410 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0635e5b9-71b2-4e23-9626-623409a24de5 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:52.171124 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1201/2401] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:44:52 2022] PUT /identity/v3/projects/ed2fc67c801448d0a68ebb8f56a1bfb6/users/d5ed29d78c5f4ac592995dc802258f19/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:44:52.182112 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b724b7-0ce5-4f10-9cb1-f0a8101faf14 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:52.182398 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b724b7-0ce5-4f10-9cb1-f0a8101faf14 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:52.182668 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b724b7-0ce5-4f10-9cb1-f0a8101faf14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:52.212822 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-38b724b7-0ce5-4f10-9cb1-f0a8101faf14 None None] MFA Rules not processed for user `d5ed29d78c5f4ac592995dc802258f19`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:52.214075 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-38b724b7-0ce5-4f10-9cb1-f0a8101faf14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:52.277614 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1201/2402] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:44:52.288790 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0635e5b9-71b2-4e23-9626-623409a24de5 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:52.291054 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0635e5b9-71b2-4e23-9626-623409a24de5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:52.347576 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0635e5b9-71b2-4e23-9626-623409a24de5 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:52.351292 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4e78f4-c8be-469b-85c5-0b32a3e42e27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:52.351892 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e78f4-c8be-469b-85c5-0b32a3e42e27 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:52.352172 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e78f4-c8be-469b-85c5-0b32a3e42e27 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:52.352442 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e78f4-c8be-469b-85c5-0b32a3e42e27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:52.361348 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4e78f4-c8be-469b-85c5-0b32a3e42e27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:52.362779 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4e78f4-c8be-469b-85c5-0b32a3e42e27 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:52.372045 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1202/2403] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:44:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:44:52.393535 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b724b7-0ce5-4f10-9cb1-f0a8101faf14 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:52.455339 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b724b7-0ce5-4f10-9cb1-f0a8101faf14 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:52.459908 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4127032-a426-4557-b1af-0ac56090e4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:52.460533 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4127032-a426-4557-b1af-0ac56090e4f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:52.460835 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4127032-a426-4557-b1af-0ac56090e4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:52.461108 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4127032-a426-4557-b1af-0ac56090e4f4 admin admin] PATH_INFO: `/v3/users/d5ed29d78c5f4ac592995dc802258f19` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:52.462228 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4127032-a426-4557-b1af-0ac56090e4f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5ed29d78c5f4ac592995dc802258f19)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:52.464151 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4127032-a426-4557-b1af-0ac56090e4f4 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:52.509173 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c4127032-a426-4557-b1af-0ac56090e4f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5ed29d78c5f4ac592995dc802258f19'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:52.515842 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c4127032-a426-4557-b1af-0ac56090e4f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5ed29d78c5f4ac592995dc802258f19'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:52.517692 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:52.529469 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c4127032-a426-4557-b1af-0ac56090e4f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5ed29d78c5f4ac592995dc802258f19'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:52.536599 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c4127032-a426-4557-b1af-0ac56090e4f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5ed29d78c5f4ac592995dc802258f19'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:52.537209 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c4127032-a426-4557-b1af-0ac56090e4f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5ed29d78c5f4ac592995dc802258f19'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:52.543506 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1202/2404] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:44:52 2022] DELETE /identity/v3/users/d5ed29d78c5f4ac592995dc802258f19 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:44:52.548009 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2be7b4-d650-474c-86c2-40908f7b43f3 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:52.548296 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2be7b4-d650-474c-86c2-40908f7b43f3 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:52.548620 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2be7b4-d650-474c-86c2-40908f7b43f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:52.568092 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-6e2be7b4-d650-474c-86c2-40908f7b43f3 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:52.569313 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6e2be7b4-d650-474c-86c2-40908f7b43f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:52.612628 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:52.628951 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1203/2405] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:44:52.638665 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4127032-a426-4557-b1af-0ac56090e4f4 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:52.640078 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c4127032-a426-4557-b1af-0ac56090e4f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:52.696224 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4127032-a426-4557-b1af-0ac56090e4f4 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:52.700364 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2fb695-7924-4836-8ddf-5ca594e403de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:52.701009 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2fb695-7924-4836-8ddf-5ca594e403de service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:52.701283 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2fb695-7924-4836-8ddf-5ca594e403de service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:52.701553 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2fb695-7924-4836-8ddf-5ca594e403de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:52.710712 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2fb695-7924-4836-8ddf-5ca594e403de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:52.712688 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2fb695-7924-4836-8ddf-5ca594e403de service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:52.722766 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1203/2406] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:44:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:44:53.004034 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2be7b4-d650-474c-86c2-40908f7b43f3 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:53.005253 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6e2be7b4-d650-474c-86c2-40908f7b43f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:53.017291 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2be7b4-d650-474c-86c2-40908f7b43f3 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:53.022249 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558347bb-ba40-4fdf-9731-58a08c4f33ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:53.022888 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558347bb-ba40-4fdf-9731-58a08c4f33ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:53.023163 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558347bb-ba40-4fdf-9731-58a08c4f33ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:53.023431 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558347bb-ba40-4fdf-9731-58a08c4f33ec admin admin] PATH_INFO: `/v3/projects/ed2fc67c801448d0a68ebb8f56a1bfb6` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:53.025101 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558347bb-ba40-4fdf-9731-58a08c4f33ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed2fc67c801448d0a68ebb8f56a1bfb6)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:53.027711 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558347bb-ba40-4fdf-9731-58a08c4f33ec admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:53.074750 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-558347bb-ba40-4fdf-9731-58a08c4f33ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed2fc67c801448d0a68ebb8f56a1bfb6', 'request_id': 'req-558347bb-ba40-4fdf-9731-58a08c4f33ec'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:53.081793 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-558347bb-ba40-4fdf-9731-58a08c4f33ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed2fc67c801448d0a68ebb8f56a1bfb6', 'request_id': 'req-558347bb-ba40-4fdf-9731-58a08c4f33ec'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:53.089569 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-558347bb-ba40-4fdf-9731-58a08c4f33ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed2fc67c801448d0a68ebb8f56a1bfb6', 'request_id': 'req-558347bb-ba40-4fdf-9731-58a08c4f33ec'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:53.100865 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-558347bb-ba40-4fdf-9731-58a08c4f33ec admin admin] The token cache is being invalidate because project ed2fc67c801448d0a68ebb8f56a1bfb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:44:53.101153 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-558347bb-ba40-4fdf-9731-58a08c4f33ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:53.101775 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-558347bb-ba40-4fdf-9731-58a08c4f33ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:53.103256 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1204/2407] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:44:53 2022] DELETE /identity/v3/projects/ed2fc67c801448d0a68ebb8f56a1bfb6 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:44:53.108035 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7813e5-1de7-4c30-8449-193201d560ac None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:53.108505 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7813e5-1de7-4c30-8449-193201d560ac None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:53.108658 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7813e5-1de7-4c30-8449-193201d560ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:53.127763 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-6a7813e5-1de7-4c30-8449-193201d560ac None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:53.128981 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6a7813e5-1de7-4c30-8449-193201d560ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:53.186969 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1204/2408] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:44:53.195709 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0cc625-237f-4cf6-8b8e-723defd23b49 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:53.195997 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0cc625-237f-4cf6-8b8e-723defd23b49 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:53.196276 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0cc625-237f-4cf6-8b8e-723defd23b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:53.216954 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-ee0cc625-237f-4cf6-8b8e-723defd23b49 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:53.218161 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ee0cc625-237f-4cf6-8b8e-723defd23b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:53.226014 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1205/2409] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:44:53.230080 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7813e5-1de7-4c30-8449-193201d560ac None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:53.240079 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7813e5-1de7-4c30-8449-193201d560ac None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:53.244832 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ce439c-46ad-4bc9-97f9-0131662f702f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:53.245464 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce439c-46ad-4bc9-97f9-0131662f702f admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:53.245764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce439c-46ad-4bc9-97f9-0131662f702f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:53.246058 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce439c-46ad-4bc9-97f9-0131662f702f admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:53.246728 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ce439c-46ad-4bc9-97f9-0131662f702f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:44:53.247029 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ce439c-46ad-4bc9-97f9-0131662f702f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:53.248803 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ce439c-46ad-4bc9-97f9-0131662f702f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:53.261126 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1205/2410] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:44:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:44:53.265457 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0cc625-237f-4cf6-8b8e-723defd23b49 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:53.279251 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0cc625-237f-4cf6-8b8e-723defd23b49 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:53.284205 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc69794b-84ad-4d6f-9648-53b9c3be0d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:53.284840 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69794b-84ad-4d6f-9648-53b9c3be0d32 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:53.285121 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69794b-84ad-4d6f-9648-53b9c3be0d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:53.285398 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69794b-84ad-4d6f-9648-53b9c3be0d32 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:53.286027 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc69794b-84ad-4d6f-9648-53b9c3be0d32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:53.287919 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc69794b-84ad-4d6f-9648-53b9c3be0d32 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:53.315394 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1206/2411] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:44:53 2022] POST /identity/v3/projects => generated 361 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:44:53.320030 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ce439c-46ad-4bc9-97f9-0131662f702f admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:53.331043 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ce439c-46ad-4bc9-97f9-0131662f702f admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:53.389920 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be292445-b14e-49b0-9657-547d7ee5622e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:53.390525 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be292445-b14e-49b0-9657-547d7ee5622e admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:53.390805 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be292445-b14e-49b0-9657-547d7ee5622e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:53.391118 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be292445-b14e-49b0-9657-547d7ee5622e admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:53.391688 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be292445-b14e-49b0-9657-547d7ee5622e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:53.393657 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be292445-b14e-49b0-9657-547d7ee5622e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:53.405509 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:53.416573 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1206/2412] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:44:53 2022] POST /identity/v3/users => generated 281 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:44:53.420816 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc69794b-84ad-4d6f-9648-53b9c3be0d32 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:53.431401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc69794b-84ad-4d6f-9648-53b9c3be0d32 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:53.435975 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4284d8a3-36af-4872-943f-22fde8097322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:53.436596 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4284d8a3-36af-4872-943f-22fde8097322 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:53.436872 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4284d8a3-36af-4872-943f-22fde8097322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:53.437141 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4284d8a3-36af-4872-943f-22fde8097322 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:53.437765 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4284d8a3-36af-4872-943f-22fde8097322 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:53.439327 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4284d8a3-36af-4872-943f-22fde8097322 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:53.449997 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1207/2413] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:44:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:44:53.453922 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be292445-b14e-49b0-9657-547d7ee5622e admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:53.463767 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be292445-b14e-49b0-9657-547d7ee5622e admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:53.468347 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856da494-77d8-4b95-9e25-b1e7438c2953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:53.469035 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856da494-77d8-4b95-9e25-b1e7438c2953 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:53.469316 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856da494-77d8-4b95-9e25-b1e7438c2953 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:53.469592 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856da494-77d8-4b95-9e25-b1e7438c2953 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:53.470222 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856da494-77d8-4b95-9e25-b1e7438c2953 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:53.471798 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856da494-77d8-4b95-9e25-b1e7438c2953 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:53.481596 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1207/2414] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:44:53 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:44:53.485586 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4284d8a3-36af-4872-943f-22fde8097322 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:53.495836 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4284d8a3-36af-4872-943f-22fde8097322 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:53.500369 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9489dd-77aa-461f-bc19-035dba145521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:53.501021 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9489dd-77aa-461f-bc19-035dba145521 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:53.501296 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9489dd-77aa-461f-bc19-035dba145521 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:53.501566 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9489dd-77aa-461f-bc19-035dba145521 admin admin] PATH_INFO: `/v3/projects/9904f2b25f684f1998fa0106e92ae6f5/users/0f9127b639da4917a7fa3566b3e139ce/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:53.516271 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9489dd-77aa-461f-bc19-035dba145521 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9904f2b25f684f1998fa0106e92ae6f5, user_id=0f9127b639da4917a7fa3566b3e139ce, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:53.518911 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9489dd-77aa-461f-bc19-035dba145521 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:53.530973 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1208/2415] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:44:53 2022] PUT /identity/v3/projects/9904f2b25f684f1998fa0106e92ae6f5/users/0f9127b639da4917a7fa3566b3e139ce/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:44:53.541050 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6082b55-ec0d-40b8-86c0-a68833da8e39 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:53.541284 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6082b55-ec0d-40b8-86c0-a68833da8e39 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:53.541560 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6082b55-ec0d-40b8-86c0-a68833da8e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:53.569654 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-c6082b55-ec0d-40b8-86c0-a68833da8e39 None None] MFA Rules not processed for user `0f9127b639da4917a7fa3566b3e139ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:53.570838 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c6082b55-ec0d-40b8-86c0-a68833da8e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:53.633415 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1208/2416] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:44:53.667660 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9489dd-77aa-461f-bc19-035dba145521 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:53.669478 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5f9489dd-77aa-461f-bc19-035dba145521 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:53.727145 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9489dd-77aa-461f-bc19-035dba145521 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:53.730799 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c5f987-e5c9-408d-a168-ab71db627de6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:53.731395 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5f987-e5c9-408d-a168-ab71db627de6 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:53.731667 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5f987-e5c9-408d-a168-ab71db627de6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:53.731934 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5f987-e5c9-408d-a168-ab71db627de6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:53.740279 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c5f987-e5c9-408d-a168-ab71db627de6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:53.741714 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c5f987-e5c9-408d-a168-ab71db627de6 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:53.751054 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1209/2417] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:44:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:44:53.767201 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6082b55-ec0d-40b8-86c0-a68833da8e39 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:53.832241 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6082b55-ec0d-40b8-86c0-a68833da8e39 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:53.837091 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a3a66b-fd49-4573-8d8f-5f83a3cedb4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:53.837750 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a3a66b-fd49-4573-8d8f-5f83a3cedb4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:53.838027 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a3a66b-fd49-4573-8d8f-5f83a3cedb4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:53.838295 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a3a66b-fd49-4573-8d8f-5f83a3cedb4a admin admin] PATH_INFO: `/v3/users/0f9127b639da4917a7fa3566b3e139ce` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:53.839430 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a3a66b-fd49-4573-8d8f-5f83a3cedb4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f9127b639da4917a7fa3566b3e139ce)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:53.841399 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a3a66b-fd49-4573-8d8f-5f83a3cedb4a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:53.903125 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-43a3a66b-fd49-4573-8d8f-5f83a3cedb4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f9127b639da4917a7fa3566b3e139ce'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:53.911135 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-43a3a66b-fd49-4573-8d8f-5f83a3cedb4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f9127b639da4917a7fa3566b3e139ce'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:53.924739 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-43a3a66b-fd49-4573-8d8f-5f83a3cedb4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f9127b639da4917a7fa3566b3e139ce'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:53.928164 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:53.935487 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-43a3a66b-fd49-4573-8d8f-5f83a3cedb4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f9127b639da4917a7fa3566b3e139ce'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:53.936637 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-43a3a66b-fd49-4573-8d8f-5f83a3cedb4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f9127b639da4917a7fa3566b3e139ce'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:53.944660 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1209/2418] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:44:53 2022] DELETE /identity/v3/users/0f9127b639da4917a7fa3566b3e139ce => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:44:53.949399 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6636c5-6681-4c74-baad-1cfe1ff0ac5c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:53.952567 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6636c5-6681-4c74-baad-1cfe1ff0ac5c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:53.953040 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6636c5-6681-4c74-baad-1cfe1ff0ac5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:53.974319 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-3c6636c5-6681-4c74-baad-1cfe1ff0ac5c None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:53.975666 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3c6636c5-6681-4c74-baad-1cfe1ff0ac5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:53.985498 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:54.045200 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1210/2419] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:44:54.055009 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a3a66b-fd49-4573-8d8f-5f83a3cedb4a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:54.057168 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-43a3a66b-fd49-4573-8d8f-5f83a3cedb4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:54.113645 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a3a66b-fd49-4573-8d8f-5f83a3cedb4a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:54.117322 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61bc25f-9a6b-40ae-9446-804c9c0f4ac5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:54.117918 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61bc25f-9a6b-40ae-9446-804c9c0f4ac5 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:54.118194 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61bc25f-9a6b-40ae-9446-804c9c0f4ac5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:54.118496 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61bc25f-9a6b-40ae-9446-804c9c0f4ac5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:54.128661 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61bc25f-9a6b-40ae-9446-804c9c0f4ac5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:54.130096 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61bc25f-9a6b-40ae-9446-804c9c0f4ac5 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:54.139592 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1210/2420] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:44:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:44:54.405409 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6636c5-6681-4c74-baad-1cfe1ff0ac5c None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:54.407035 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3c6636c5-6681-4c74-baad-1cfe1ff0ac5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:54.421125 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6636c5-6681-4c74-baad-1cfe1ff0ac5c None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:54.427040 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78017ce0-267e-41c9-b620-aad0680b1708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:54.427682 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78017ce0-267e-41c9-b620-aad0680b1708 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:54.427960 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78017ce0-267e-41c9-b620-aad0680b1708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:54.428232 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78017ce0-267e-41c9-b620-aad0680b1708 admin admin] PATH_INFO: `/v3/projects/9904f2b25f684f1998fa0106e92ae6f5` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:54.429397 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78017ce0-267e-41c9-b620-aad0680b1708 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9904f2b25f684f1998fa0106e92ae6f5)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:54.431549 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78017ce0-267e-41c9-b620-aad0680b1708 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:54.487375 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-78017ce0-267e-41c9-b620-aad0680b1708 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9904f2b25f684f1998fa0106e92ae6f5', 'request_id': 'req-78017ce0-267e-41c9-b620-aad0680b1708'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:54.496604 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-78017ce0-267e-41c9-b620-aad0680b1708 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9904f2b25f684f1998fa0106e92ae6f5', 'request_id': 'req-78017ce0-267e-41c9-b620-aad0680b1708'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:54.512825 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-78017ce0-267e-41c9-b620-aad0680b1708 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9904f2b25f684f1998fa0106e92ae6f5', 'request_id': 'req-78017ce0-267e-41c9-b620-aad0680b1708'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:54.534387 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-78017ce0-267e-41c9-b620-aad0680b1708 admin admin] The token cache is being invalidate because project 9904f2b25f684f1998fa0106e92ae6f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:44:54.534590 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-78017ce0-267e-41c9-b620-aad0680b1708 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:54.535524 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-78017ce0-267e-41c9-b620-aad0680b1708 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:44:54.544590 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1211/2421] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:44:54 2022] DELETE /identity/v3/projects/9904f2b25f684f1998fa0106e92ae6f5 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:44:54.546757 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515452f1-5da5-4d61-bf81-a5dc4de3a949 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:54.547484 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515452f1-5da5-4d61-bf81-a5dc4de3a949 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:54.547921 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515452f1-5da5-4d61-bf81-a5dc4de3a949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:54.583507 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-515452f1-5da5-4d61-bf81-a5dc4de3a949 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:54.584657 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-515452f1-5da5-4d61-bf81-a5dc4de3a949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:54.659427 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1211/2422] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:44:54.666701 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db39893-4176-4767-a66c-c9afa7634a52 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:54.666974 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db39893-4176-4767-a66c-c9afa7634a52 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:54.667245 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db39893-4176-4767-a66c-c9afa7634a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:54.687504 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-0db39893-4176-4767-a66c-c9afa7634a52 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:54.688742 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0db39893-4176-4767-a66c-c9afa7634a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:54.698003 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1212/2423] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:44:54.702593 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515452f1-5da5-4d61-bf81-a5dc4de3a949 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:54.712410 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515452f1-5da5-4d61-bf81-a5dc4de3a949 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:54.717617 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4ac5a5-c8e8-4a24-9c2f-95d76f267cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:54.718239 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ac5a5-c8e8-4a24-9c2f-95d76f267cb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:54.718492 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ac5a5-c8e8-4a24-9c2f-95d76f267cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:54.718758 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ac5a5-c8e8-4a24-9c2f-95d76f267cb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:54.719393 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4ac5a5-c8e8-4a24-9c2f-95d76f267cb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:44:54.719690 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4ac5a5-c8e8-4a24-9c2f-95d76f267cb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:54.721325 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4ac5a5-c8e8-4a24-9c2f-95d76f267cb3 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:54.733218 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1212/2424] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:44:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:44:54.737407 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db39893-4176-4767-a66c-c9afa7634a52 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:54.747273 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db39893-4176-4767-a66c-c9afa7634a52 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:54.751961 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4da776-5450-45c5-a7d2-4982fb516ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:54.752597 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4da776-5450-45c5-a7d2-4982fb516ae5 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:54.752875 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4da776-5450-45c5-a7d2-4982fb516ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:54.753143 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4da776-5450-45c5-a7d2-4982fb516ae5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:54.753772 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4da776-5450-45c5-a7d2-4982fb516ae5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:54.755658 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4da776-5450-45c5-a7d2-4982fb516ae5 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:54.783604 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1213/2425] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:44:54 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:44:54.787764 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4ac5a5-c8e8-4a24-9c2f-95d76f267cb3 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:54.797909 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4ac5a5-c8e8-4a24-9c2f-95d76f267cb3 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:54.853458 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760647a-fe8f-4dd8-948e-853f7586c904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:54.854063 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760647a-fe8f-4dd8-948e-853f7586c904 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:54.854343 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760647a-fe8f-4dd8-948e-853f7586c904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:54.854620 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760647a-fe8f-4dd8-948e-853f7586c904 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:54.855252 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9760647a-fe8f-4dd8-948e-853f7586c904 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:54.857230 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9760647a-fe8f-4dd8-948e-853f7586c904 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:54.877774 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1213/2426] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:44:54 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:44:54.882021 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4da776-5450-45c5-a7d2-4982fb516ae5 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:54.892466 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4da776-5450-45c5-a7d2-4982fb516ae5 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:54.896975 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ae532f-c26f-4846-8297-6bad74e3cafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:54.897562 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ae532f-c26f-4846-8297-6bad74e3cafc admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:54.897832 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ae532f-c26f-4846-8297-6bad74e3cafc admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:54.898097 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ae532f-c26f-4846-8297-6bad74e3cafc admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:54.898716 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ae532f-c26f-4846-8297-6bad74e3cafc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:54.900920 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ae532f-c26f-4846-8297-6bad74e3cafc admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:54.911033 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1214/2427] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:44:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:44:54.914899 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760647a-fe8f-4dd8-948e-853f7586c904 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:54.924727 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760647a-fe8f-4dd8-948e-853f7586c904 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:54.929269 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da981e-eb96-48b4-8f65-317e523ca12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:54.929856 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da981e-eb96-48b4-8f65-317e523ca12a admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:54.930134 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da981e-eb96-48b4-8f65-317e523ca12a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:54.930405 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da981e-eb96-48b4-8f65-317e523ca12a admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:54.931029 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00da981e-eb96-48b4-8f65-317e523ca12a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:54.933485 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00da981e-eb96-48b4-8f65-317e523ca12a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:54.944589 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1214/2428] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:44:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:44:54.949709 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ae532f-c26f-4846-8297-6bad74e3cafc admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:54.959742 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ae532f-c26f-4846-8297-6bad74e3cafc admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:54.964480 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76988d83-c8b1-493e-a5f7-1e8a8808da42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:54.965156 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76988d83-c8b1-493e-a5f7-1e8a8808da42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:54.965434 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76988d83-c8b1-493e-a5f7-1e8a8808da42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:54.965707 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76988d83-c8b1-493e-a5f7-1e8a8808da42 admin admin] PATH_INFO: `/v3/projects/119962d798ca45c3b33509d149e56aa8/users/f2e9e06dd9af4bb98bca3fd7546280a3/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:54.979943 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76988d83-c8b1-493e-a5f7-1e8a8808da42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=119962d798ca45c3b33509d149e56aa8, user_id=f2e9e06dd9af4bb98bca3fd7546280a3, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:54.982639 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76988d83-c8b1-493e-a5f7-1e8a8808da42 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:54.996127 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1215/2429] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:44:54 2022] PUT /identity/v3/projects/119962d798ca45c3b33509d149e56aa8/users/f2e9e06dd9af4bb98bca3fd7546280a3/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:44:55.060745 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106d393f-6ddb-4ac2-977c-35766bc3a02d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:55.061025 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106d393f-6ddb-4ac2-977c-35766bc3a02d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:55.061289 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106d393f-6ddb-4ac2-977c-35766bc3a02d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:55.068638 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:55.093803 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-106d393f-6ddb-4ac2-977c-35766bc3a02d None None] MFA Rules not processed for user `f2e9e06dd9af4bb98bca3fd7546280a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:55.094994 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-106d393f-6ddb-4ac2-977c-35766bc3a02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:55.164113 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1215/2430] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:44:55.165357 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76988d83-c8b1-493e-a5f7-1e8a8808da42 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:55.201467 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:55.230683 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76988d83-c8b1-493e-a5f7-1e8a8808da42 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:55.235482 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb25d5b-1597-4ab7-96a0-843baa5bf1c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:55.236080 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb25d5b-1597-4ab7-96a0-843baa5bf1c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:55.236358 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb25d5b-1597-4ab7-96a0-843baa5bf1c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:55.236660 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb25d5b-1597-4ab7-96a0-843baa5bf1c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:55.237269 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb25d5b-1597-4ab7-96a0-843baa5bf1c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:55.239304 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb25d5b-1597-4ab7-96a0-843baa5bf1c8 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:55.280331 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1216/2431] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:44:55 2022] POST /identity/v3/projects => generated 383 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:44:55.284926 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106d393f-6ddb-4ac2-977c-35766bc3a02d None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:55.298741 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106d393f-6ddb-4ac2-977c-35766bc3a02d None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:55.366961 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda568f0-f54f-400c-b043-9ed3574527c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:55.367576 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda568f0-f54f-400c-b043-9ed3574527c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:55.367855 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda568f0-f54f-400c-b043-9ed3574527c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:55.368127 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda568f0-f54f-400c-b043-9ed3574527c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:55.368787 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda568f0-f54f-400c-b043-9ed3574527c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:55.371311 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda568f0-f54f-400c-b043-9ed3574527c8 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:55.391765 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1216/2432] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:44:55 2022] POST /identity/v3/users => generated 292 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:44:55.395773 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb25d5b-1597-4ab7-96a0-843baa5bf1c8 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:55.406080 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb25d5b-1597-4ab7-96a0-843baa5bf1c8 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:55.410860 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b03c4b-3754-467c-8cca-8ed7874b7488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:55.411450 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b03c4b-3754-467c-8cca-8ed7874b7488 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:55.411751 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b03c4b-3754-467c-8cca-8ed7874b7488 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:55.412037 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b03c4b-3754-467c-8cca-8ed7874b7488 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:55.412705 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b03c4b-3754-467c-8cca-8ed7874b7488 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:55.414349 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b03c4b-3754-467c-8cca-8ed7874b7488 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:55.425397 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1217/2433] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:44:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:44:55.429264 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda568f0-f54f-400c-b043-9ed3574527c8 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:55.439737 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda568f0-f54f-400c-b043-9ed3574527c8 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:55.444371 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e695675-3784-427b-89ae-7ee311bcf457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:55.445007 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e695675-3784-427b-89ae-7ee311bcf457 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:55.445286 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e695675-3784-427b-89ae-7ee311bcf457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:55.445557 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e695675-3784-427b-89ae-7ee311bcf457 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:55.446201 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e695675-3784-427b-89ae-7ee311bcf457 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:55.448063 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e695675-3784-427b-89ae-7ee311bcf457 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:55.457893 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1217/2434] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:44:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:44:55.462095 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b03c4b-3754-467c-8cca-8ed7874b7488 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:55.472455 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b03c4b-3754-467c-8cca-8ed7874b7488 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:55.477422 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b8e85-8902-497c-8fb7-7722a0facf43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:55.478075 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b8e85-8902-497c-8fb7-7722a0facf43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:55.478864 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b8e85-8902-497c-8fb7-7722a0facf43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:55.479135 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b8e85-8902-497c-8fb7-7722a0facf43 admin admin] PATH_INFO: `/v3/projects/ff17aaa61e444a96a17cbd83402f7742/users/9f89a249d5574f919d162f417b71b1de/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:55.494605 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98b8e85-8902-497c-8fb7-7722a0facf43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff17aaa61e444a96a17cbd83402f7742, user_id=9f89a249d5574f919d162f417b71b1de, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:55.497285 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98b8e85-8902-497c-8fb7-7722a0facf43 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:55.509828 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1218/2435] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:44:55 2022] PUT /identity/v3/projects/ff17aaa61e444a96a17cbd83402f7742/users/9f89a249d5574f919d162f417b71b1de/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:44:55.519997 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87834fe-6f90-4ae4-a5e5-db07dce400ee None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:55.520279 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87834fe-6f90-4ae4-a5e5-db07dce400ee None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:55.520572 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87834fe-6f90-4ae4-a5e5-db07dce400ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:55.547832 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b87834fe-6f90-4ae4-a5e5-db07dce400ee None None] MFA Rules not processed for user `9f89a249d5574f919d162f417b71b1de`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:55.549075 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b87834fe-6f90-4ae4-a5e5-db07dce400ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:55.608595 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:55.612629 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1218/2436] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:44:55.618228 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b8e85-8902-497c-8fb7-7722a0facf43 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:55.684082 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b8e85-8902-497c-8fb7-7722a0facf43 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:55.688748 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd45033-74a7-4858-bfe7-e9795f3ad554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:55.689359 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd45033-74a7-4858-bfe7-e9795f3ad554 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:55.689639 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd45033-74a7-4858-bfe7-e9795f3ad554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:55.690036 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd45033-74a7-4858-bfe7-e9795f3ad554 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:55.690682 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd45033-74a7-4858-bfe7-e9795f3ad554 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:55.692623 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd45033-74a7-4858-bfe7-e9795f3ad554 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:55.722906 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1219/2437] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:44:55 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:44:55.724118 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87834fe-6f90-4ae4-a5e5-db07dce400ee None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:55.735290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87834fe-6f90-4ae4-a5e5-db07dce400ee None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:55.793856 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6340c3-cf30-4d57-aabd-6c1e67bbeca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:55.794463 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6340c3-cf30-4d57-aabd-6c1e67bbeca8 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:55.794738 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6340c3-cf30-4d57-aabd-6c1e67bbeca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:55.795003 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6340c3-cf30-4d57-aabd-6c1e67bbeca8 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:55.795621 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6340c3-cf30-4d57-aabd-6c1e67bbeca8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:55.797897 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6340c3-cf30-4d57-aabd-6c1e67bbeca8 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:55.818430 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1219/2438] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:44:55 2022] POST /identity/v3/users => generated 298 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:44:55.822909 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd45033-74a7-4858-bfe7-e9795f3ad554 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:55.833140 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd45033-74a7-4858-bfe7-e9795f3ad554 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:55.838206 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea0df49-8744-4e1c-9f25-18d05278871e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:55.838791 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea0df49-8744-4e1c-9f25-18d05278871e admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:55.839068 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea0df49-8744-4e1c-9f25-18d05278871e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:55.839332 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea0df49-8744-4e1c-9f25-18d05278871e admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:55.839963 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea0df49-8744-4e1c-9f25-18d05278871e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:55.841641 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea0df49-8744-4e1c-9f25-18d05278871e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:55.852812 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1220/2439] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:44:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 09:44:55.857402 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6340c3-cf30-4d57-aabd-6c1e67bbeca8 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:55.867055 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6340c3-cf30-4d57-aabd-6c1e67bbeca8 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:55.871714 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f5a69-061e-4ad5-a6bd-75904562d73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:55.872303 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f5a69-061e-4ad5-a6bd-75904562d73a admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:55.872611 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f5a69-061e-4ad5-a6bd-75904562d73a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:55.872881 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f5a69-061e-4ad5-a6bd-75904562d73a admin admin] PATH_INFO: `/v3/projects/a7bdd1e242f041c4bf0a9e45892c046a/users/c9e820504d3d42e49d42d216210f5edd/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:55.888195 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759f5a69-061e-4ad5-a6bd-75904562d73a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7bdd1e242f041c4bf0a9e45892c046a, user_id=c9e820504d3d42e49d42d216210f5edd, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:55.890882 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759f5a69-061e-4ad5-a6bd-75904562d73a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:55.904188 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1220/2440] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:44:55 2022] PUT /identity/v3/projects/a7bdd1e242f041c4bf0a9e45892c046a/users/c9e820504d3d42e49d42d216210f5edd/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:44:55.908427 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea0df49-8744-4e1c-9f25-18d05278871e admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:55.918970 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea0df49-8744-4e1c-9f25-18d05278871e admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:55.969949 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:55.976755 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f0691-8b03-49dc-8d34-9fd9a603fd47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:55.977386 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f0691-8b03-49dc-8d34-9fd9a603fd47 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:55.977791 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f0691-8b03-49dc-8d34-9fd9a603fd47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:55.978068 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f0691-8b03-49dc-8d34-9fd9a603fd47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:55.979261 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295f0691-8b03-49dc-8d34-9fd9a603fd47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:55.980956 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295f0691-8b03-49dc-8d34-9fd9a603fd47 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:55.990798 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1221/2441] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:44:55 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:44:55.995027 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f5a69-061e-4ad5-a6bd-75904562d73a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:56.005142 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f5a69-061e-4ad5-a6bd-75904562d73a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:56.010240 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e57f66a-6b9a-4c78-99c1-11d7822c3804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:56.010880 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e57f66a-6b9a-4c78-99c1-11d7822c3804 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:56.011156 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e57f66a-6b9a-4c78-99c1-11d7822c3804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:56.011427 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e57f66a-6b9a-4c78-99c1-11d7822c3804 admin admin] PATH_INFO: `/v3/projects/a7bdd1e242f041c4bf0a9e45892c046a/users/c9e820504d3d42e49d42d216210f5edd/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:56.013426 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e57f66a-6b9a-4c78-99c1-11d7822c3804 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7bdd1e242f041c4bf0a9e45892c046a, user_id=c9e820504d3d42e49d42d216210f5edd, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:56.016051 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e57f66a-6b9a-4c78-99c1-11d7822c3804 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:56.026611 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1221/2442] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:44:55 2022] PUT /identity/v3/projects/a7bdd1e242f041c4bf0a9e45892c046a/users/c9e820504d3d42e49d42d216210f5edd/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:44:56.036633 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c8c68-5982-4729-a69b-da5bd74de95e None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:56.036923 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c8c68-5982-4729-a69b-da5bd74de95e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:56.037197 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c8c68-5982-4729-a69b-da5bd74de95e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:56.064406 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-ad5c8c68-5982-4729-a69b-da5bd74de95e None None] MFA Rules not processed for user `c9e820504d3d42e49d42d216210f5edd`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:44:56.065659 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ad5c8c68-5982-4729-a69b-da5bd74de95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:56.132808 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1222/2443] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:44:56 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:44:56.144935 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e57f66a-6b9a-4c78-99c1-11d7822c3804 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:56.146615 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2e57f66a-6b9a-4c78-99c1-11d7822c3804 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:56.206326 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e57f66a-6b9a-4c78-99c1-11d7822c3804 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:56.210124 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35ffa65-9407-4ace-b985-41ce7b7f7d8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:56.210723 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ffa65-9407-4ace-b985-41ce7b7f7d8e service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:56.211003 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ffa65-9407-4ace-b985-41ce7b7f7d8e service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:56.211321 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ffa65-9407-4ace-b985-41ce7b7f7d8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:56.266521 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35ffa65-9407-4ace-b985-41ce7b7f7d8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:56.267991 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35ffa65-9407-4ace-b985-41ce7b7f7d8e service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:56.277649 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1222/2444] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:44:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:44:58.843527 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5c8c68-5982-4729-a69b-da5bd74de95e None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:58.862372 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5c8c68-5982-4729-a69b-da5bd74de95e None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:58.865920 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe1e87-ba6b-4ce3-91af-e0bda725d0a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:58.866510 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe1e87-ba6b-4ce3-91af-e0bda725d0a4 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:58.866783 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe1e87-ba6b-4ce3-91af-e0bda725d0a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:58.867052 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe1e87-ba6b-4ce3-91af-e0bda725d0a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:58.924330 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebe1e87-ba6b-4ce3-91af-e0bda725d0a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:58.925715 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebe1e87-ba6b-4ce3-91af-e0bda725d0a4 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:58.936087 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1223/2445] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:44:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:44:59.500545 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35ffa65-9407-4ace-b985-41ce7b7f7d8e service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:44:59.502206 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b35ffa65-9407-4ace-b985-41ce7b7f7d8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:44:59.514567 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:44:59.541554 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35ffa65-9407-4ace-b985-41ce7b7f7d8e service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:44:59.545143 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566df15c-caf9-471b-b431-5a8b0be76bc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:44:59.545763 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566df15c-caf9-471b-b431-5a8b0be76bc1 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:44:59.546039 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566df15c-caf9-471b-b431-5a8b0be76bc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:44:59.546302 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566df15c-caf9-471b-b431-5a8b0be76bc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:44:59.562161 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566df15c-caf9-471b-b431-5a8b0be76bc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:44:59.564919 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566df15c-caf9-471b-b431-5a8b0be76bc1 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:44:59.580637 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1223/2446] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:44:59 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:45:06.702912 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe1e87-ba6b-4ce3-91af-e0bda725d0a4 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:06.766894 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe1e87-ba6b-4ce3-91af-e0bda725d0a4 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:06.771619 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bebabf5-cf91-4fc6-8bb8-9dc4730ee8a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:06.772389 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bebabf5-cf91-4fc6-8bb8-9dc4730ee8a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:06.772732 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bebabf5-cf91-4fc6-8bb8-9dc4730ee8a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:06.773076 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bebabf5-cf91-4fc6-8bb8-9dc4730ee8a7 admin admin] PATH_INFO: `/v3/users/f2e9e06dd9af4bb98bca3fd7546280a3` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:06.774298 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bebabf5-cf91-4fc6-8bb8-9dc4730ee8a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2e9e06dd9af4bb98bca3fd7546280a3)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:06.776292 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bebabf5-cf91-4fc6-8bb8-9dc4730ee8a7 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:06.789904 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:45:06.829785 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0bebabf5-cf91-4fc6-8bb8-9dc4730ee8a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2e9e06dd9af4bb98bca3fd7546280a3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:06.836716 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0bebabf5-cf91-4fc6-8bb8-9dc4730ee8a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2e9e06dd9af4bb98bca3fd7546280a3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:06.848824 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0bebabf5-cf91-4fc6-8bb8-9dc4730ee8a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2e9e06dd9af4bb98bca3fd7546280a3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:06.855792 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0bebabf5-cf91-4fc6-8bb8-9dc4730ee8a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2e9e06dd9af4bb98bca3fd7546280a3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:06.856397 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0bebabf5-cf91-4fc6-8bb8-9dc4730ee8a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2e9e06dd9af4bb98bca3fd7546280a3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:06.862202 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1224/2447] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:45:06 2022] DELETE /identity/v3/users/f2e9e06dd9af4bb98bca3fd7546280a3 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:45:06.866793 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e51a8-358c-4678-ba5f-cc9a27cdde51 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:06.867289 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e51a8-358c-4678-ba5f-cc9a27cdde51 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:06.867388 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e51a8-358c-4678-ba5f-cc9a27cdde51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:06.887335 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-ca8e51a8-358c-4678-ba5f-cc9a27cdde51 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:45:06.888540 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ca8e51a8-358c-4678-ba5f-cc9a27cdde51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:06.945274 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1224/2448] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:45:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:45:06.955776 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bebabf5-cf91-4fc6-8bb8-9dc4730ee8a7 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:06.957287 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0bebabf5-cf91-4fc6-8bb8-9dc4730ee8a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:07.014657 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bebabf5-cf91-4fc6-8bb8-9dc4730ee8a7 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:07.018365 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec757cef-e302-448c-a0a4-316cf6f7db18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:07.018966 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec757cef-e302-448c-a0a4-316cf6f7db18 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:07.019245 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec757cef-e302-448c-a0a4-316cf6f7db18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:07.019515 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec757cef-e302-448c-a0a4-316cf6f7db18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:07.028507 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec757cef-e302-448c-a0a4-316cf6f7db18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:07.029942 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec757cef-e302-448c-a0a4-316cf6f7db18 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:07.039249 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1225/2449] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:45:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:45:07.306091 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8e51a8-358c-4678-ba5f-cc9a27cdde51 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:07.307579 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ca8e51a8-358c-4678-ba5f-cc9a27cdde51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:07.322541 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8e51a8-358c-4678-ba5f-cc9a27cdde51 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:07.327943 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b3817e-f0cd-48e1-9e41-89e7b2d66931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:07.328877 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3817e-f0cd-48e1-9e41-89e7b2d66931 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:07.329046 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3817e-f0cd-48e1-9e41-89e7b2d66931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:07.329318 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3817e-f0cd-48e1-9e41-89e7b2d66931 admin admin] PATH_INFO: `/v3/projects/119962d798ca45c3b33509d149e56aa8` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:07.330449 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b3817e-f0cd-48e1-9e41-89e7b2d66931 admin admin] RBAC: Authorizing `identity:delete_project(project_id=119962d798ca45c3b33509d149e56aa8)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:07.332372 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b3817e-f0cd-48e1-9e41-89e7b2d66931 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:07.379490 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-09b3817e-f0cd-48e1-9e41-89e7b2d66931 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '119962d798ca45c3b33509d149e56aa8', 'request_id': 'req-09b3817e-f0cd-48e1-9e41-89e7b2d66931'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:07.385648 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-09b3817e-f0cd-48e1-9e41-89e7b2d66931 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '119962d798ca45c3b33509d149e56aa8', 'request_id': 'req-09b3817e-f0cd-48e1-9e41-89e7b2d66931'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:07.391286 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-09b3817e-f0cd-48e1-9e41-89e7b2d66931 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '119962d798ca45c3b33509d149e56aa8', 'request_id': 'req-09b3817e-f0cd-48e1-9e41-89e7b2d66931'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:07.403556 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-09b3817e-f0cd-48e1-9e41-89e7b2d66931 admin admin] The token cache is being invalidate because project 119962d798ca45c3b33509d149e56aa8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:45:07.403853 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-09b3817e-f0cd-48e1-9e41-89e7b2d66931 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:07.404448 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-09b3817e-f0cd-48e1-9e41-89e7b2d66931 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:07.406057 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1225/2450] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:45:07 2022] DELETE /identity/v3/projects/119962d798ca45c3b33509d149e56aa8 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:45:07.410099 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec757cef-e302-448c-a0a4-316cf6f7db18 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:07.419985 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec757cef-e302-448c-a0a4-316cf6f7db18 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:07.476808 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5889151-2438-404c-9551-b6ee66df227e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:07.477451 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5889151-2438-404c-9551-b6ee66df227e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:07.477754 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5889151-2438-404c-9551-b6ee66df227e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:07.478019 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5889151-2438-404c-9551-b6ee66df227e admin admin] PATH_INFO: `/v3/users/9f89a249d5574f919d162f417b71b1de` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:07.479116 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5889151-2438-404c-9551-b6ee66df227e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f89a249d5574f919d162f417b71b1de)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:07.481626 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5889151-2438-404c-9551-b6ee66df227e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:07.515752 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:45:07.528860 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e5889151-2438-404c-9551-b6ee66df227e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f89a249d5574f919d162f417b71b1de'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:07.535668 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e5889151-2438-404c-9551-b6ee66df227e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f89a249d5574f919d162f417b71b1de'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:07.548060 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e5889151-2438-404c-9551-b6ee66df227e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f89a249d5574f919d162f417b71b1de'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:07.554904 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e5889151-2438-404c-9551-b6ee66df227e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f89a249d5574f919d162f417b71b1de'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:07.555512 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e5889151-2438-404c-9551-b6ee66df227e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f89a249d5574f919d162f417b71b1de'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:07.561884 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1226/2451] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:45:07 2022] DELETE /identity/v3/users/9f89a249d5574f919d162f417b71b1de => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:45:07.831116 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b3817e-f0cd-48e1-9e41-89e7b2d66931 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:07.833330 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-09b3817e-f0cd-48e1-9e41-89e7b2d66931 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:07.850698 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b3817e-f0cd-48e1-9e41-89e7b2d66931 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:07.903285 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:45:07.909735 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3aec92-5e4c-4c54-9f5b-50a601980b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:07.910688 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3aec92-5e4c-4c54-9f5b-50a601980b9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:07.911027 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3aec92-5e4c-4c54-9f5b-50a601980b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:07.911301 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3aec92-5e4c-4c54-9f5b-50a601980b9b admin admin] PATH_INFO: `/v3/projects/ff17aaa61e444a96a17cbd83402f7742` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:07.912728 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3aec92-5e4c-4c54-9f5b-50a601980b9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff17aaa61e444a96a17cbd83402f7742)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:07.914815 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3aec92-5e4c-4c54-9f5b-50a601980b9b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:07.960750 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5a3aec92-5e4c-4c54-9f5b-50a601980b9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff17aaa61e444a96a17cbd83402f7742', 'request_id': 'req-5a3aec92-5e4c-4c54-9f5b-50a601980b9b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:07.967460 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5a3aec92-5e4c-4c54-9f5b-50a601980b9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff17aaa61e444a96a17cbd83402f7742', 'request_id': 'req-5a3aec92-5e4c-4c54-9f5b-50a601980b9b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:07.973168 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5a3aec92-5e4c-4c54-9f5b-50a601980b9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff17aaa61e444a96a17cbd83402f7742', 'request_id': 'req-5a3aec92-5e4c-4c54-9f5b-50a601980b9b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:07.985362 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5a3aec92-5e4c-4c54-9f5b-50a601980b9b admin admin] The token cache is being invalidate because project ff17aaa61e444a96a17cbd83402f7742 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:45:07.985652 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5a3aec92-5e4c-4c54-9f5b-50a601980b9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:07.986272 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5a3aec92-5e4c-4c54-9f5b-50a601980b9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:07.987761 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1226/2452] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:45:07 2022] DELETE /identity/v3/projects/ff17aaa61e444a96a17cbd83402f7742 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:45:07.991864 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5889151-2438-404c-9551-b6ee66df227e admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:07.993276 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e5889151-2438-404c-9551-b6ee66df227e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:08.005642 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5889151-2438-404c-9551-b6ee66df227e admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:08.061605 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f68401-705a-4992-9c01-48f0368851dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:08.062259 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f68401-705a-4992-9c01-48f0368851dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:08.062539 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f68401-705a-4992-9c01-48f0368851dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:08.062807 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f68401-705a-4992-9c01-48f0368851dc admin admin] PATH_INFO: `/v3/users/c9e820504d3d42e49d42d216210f5edd` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:08.064017 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f68401-705a-4992-9c01-48f0368851dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9e820504d3d42e49d42d216210f5edd)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:08.065960 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f68401-705a-4992-9c01-48f0368851dc admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:08.111486 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-67f68401-705a-4992-9c01-48f0368851dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9e820504d3d42e49d42d216210f5edd'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:08.118372 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-67f68401-705a-4992-9c01-48f0368851dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9e820504d3d42e49d42d216210f5edd'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:08.130677 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-67f68401-705a-4992-9c01-48f0368851dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9e820504d3d42e49d42d216210f5edd'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:08.138201 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-67f68401-705a-4992-9c01-48f0368851dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9e820504d3d42e49d42d216210f5edd'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:08.138907 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-67f68401-705a-4992-9c01-48f0368851dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9e820504d3d42e49d42d216210f5edd'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:08.144674 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1227/2453] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:45:07 2022] DELETE /identity/v3/users/c9e820504d3d42e49d42d216210f5edd => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:45:08.410386 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3aec92-5e4c-4c54-9f5b-50a601980b9b admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:08.411838 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5a3aec92-5e4c-4c54-9f5b-50a601980b9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:08.426795 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3aec92-5e4c-4c54-9f5b-50a601980b9b admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:08.483589 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e440099-4ae3-4f73-a563-ec2fe21e56f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:08.484210 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e440099-4ae3-4f73-a563-ec2fe21e56f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:08.484486 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e440099-4ae3-4f73-a563-ec2fe21e56f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:08.484784 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e440099-4ae3-4f73-a563-ec2fe21e56f2 admin admin] PATH_INFO: `/v3/projects/a7bdd1e242f041c4bf0a9e45892c046a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:08.485883 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e440099-4ae3-4f73-a563-ec2fe21e56f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7bdd1e242f041c4bf0a9e45892c046a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:08.487771 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e440099-4ae3-4f73-a563-ec2fe21e56f2 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:08.535324 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6e440099-4ae3-4f73-a563-ec2fe21e56f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7bdd1e242f041c4bf0a9e45892c046a', 'request_id': 'req-6e440099-4ae3-4f73-a563-ec2fe21e56f2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:08.541736 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6e440099-4ae3-4f73-a563-ec2fe21e56f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7bdd1e242f041c4bf0a9e45892c046a', 'request_id': 'req-6e440099-4ae3-4f73-a563-ec2fe21e56f2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:08.548037 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6e440099-4ae3-4f73-a563-ec2fe21e56f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7bdd1e242f041c4bf0a9e45892c046a', 'request_id': 'req-6e440099-4ae3-4f73-a563-ec2fe21e56f2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:08.560251 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6e440099-4ae3-4f73-a563-ec2fe21e56f2 admin admin] The token cache is being invalidate because project a7bdd1e242f041c4bf0a9e45892c046a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:45:08.560559 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6e440099-4ae3-4f73-a563-ec2fe21e56f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:08.561287 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6e440099-4ae3-4f73-a563-ec2fe21e56f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:08.562779 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1227/2454] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:45:08 2022] DELETE /identity/v3/projects/a7bdd1e242f041c4bf0a9e45892c046a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:45:10.454446 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053fa64-5b92-4eda-879e-4c341d5aa189 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:10.454903 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053fa64-5b92-4eda-879e-4c341d5aa189 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:10.454999 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053fa64-5b92-4eda-879e-4c341d5aa189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:10.484401 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-4053fa64-5b92-4eda-879e-4c341d5aa189 None None] MFA Rules not processed for user `7f389510a6a044d8a7487561b169544b`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:45:10.485560 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4053fa64-5b92-4eda-879e-4c341d5aa189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:10.498369 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:45:10.549946 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1228/2455] 10.222.0.70 () {58 vars in 1009 bytes} [Thu Oct 27 09:45:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:45:10.597789 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e440099-4ae3-4f73-a563-ec2fe21e56f2 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:10.599183 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6e440099-4ae3-4f73-a563-ec2fe21e56f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:10.619149 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:45:10.672198 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e440099-4ae3-4f73-a563-ec2fe21e56f2 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:10.677264 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4873c48-d992-4570-9e72-9f45b59bf181 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eecf79ea4700496c89b1a67c614b34d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:10.677866 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4873c48-d992-4570-9e72-9f45b59bf181 service swift] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:10.678167 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4873c48-d992-4570-9e72-9f45b59bf181 service swift] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:10.678438 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4873c48-d992-4570-9e72-9f45b59bf181 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:10.686962 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4873c48-d992-4570-9e72-9f45b59bf181 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:10.688574 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4873c48-d992-4570-9e72-9f45b59bf181 service swift] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:10.696456 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1228/2456] 10.222.0.70 () {60 vars in 1384 bytes} [Thu Oct 27 09:45:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 27 09:45:11.022133 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4053fa64-5b92-4eda-879e-4c341d5aa189 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:11.023572 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4053fa64-5b92-4eda-879e-4c341d5aa189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:11.108360 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4053fa64-5b92-4eda-879e-4c341d5aa189 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:11.114242 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff70fca-cb3f-45e1-8ca0-e8d99dac91dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:11.115185 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff70fca-cb3f-45e1-8ca0-e8d99dac91dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:11.115464 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff70fca-cb3f-45e1-8ca0-e8d99dac91dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:11.115793 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff70fca-cb3f-45e1-8ca0-e8d99dac91dc admin admin] PATH_INFO: `/v3/users/f1c5959d00a74e16b9ec23ed075a4731` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:11.117240 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff70fca-cb3f-45e1-8ca0-e8d99dac91dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1c5959d00a74e16b9ec23ed075a4731)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:11.119941 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff70fca-cb3f-45e1-8ca0-e8d99dac91dc admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:11.172117 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6ff70fca-cb3f-45e1-8ca0-e8d99dac91dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1c5959d00a74e16b9ec23ed075a4731'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:11.182872 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6ff70fca-cb3f-45e1-8ca0-e8d99dac91dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1c5959d00a74e16b9ec23ed075a4731'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:11.199477 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6ff70fca-cb3f-45e1-8ca0-e8d99dac91dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1c5959d00a74e16b9ec23ed075a4731'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:11.209213 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6ff70fca-cb3f-45e1-8ca0-e8d99dac91dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1c5959d00a74e16b9ec23ed075a4731'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:11.210076 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6ff70fca-cb3f-45e1-8ca0-e8d99dac91dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1c5959d00a74e16b9ec23ed075a4731'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:11.218288 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1229/2457] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:45:11 2022] DELETE /identity/v3/users/f1c5959d00a74e16b9ec23ed075a4731 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:45:11.223149 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad94a6e-19cf-4795-a980-f2e38715d9f6 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:11.223583 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad94a6e-19cf-4795-a980-f2e38715d9f6 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:11.223865 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad94a6e-19cf-4795-a980-f2e38715d9f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:11.247145 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-cad94a6e-19cf-4795-a980-f2e38715d9f6 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:45:11.248365 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-cad94a6e-19cf-4795-a980-f2e38715d9f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:11.320562 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1229/2458] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:45:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:45:11.330758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff70fca-cb3f-45e1-8ca0-e8d99dac91dc admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:11.332216 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6ff70fca-cb3f-45e1-8ca0-e8d99dac91dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:11.380763 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:45:11.400867 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff70fca-cb3f-45e1-8ca0-e8d99dac91dc admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:11.405098 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631c9c02-f565-4fc9-8c74-1eee3538d7b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:11.405706 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c9c02-f565-4fc9-8c74-1eee3538d7b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:11.406024 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c9c02-f565-4fc9-8c74-1eee3538d7b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:11.406245 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c9c02-f565-4fc9-8c74-1eee3538d7b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:11.416143 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631c9c02-f565-4fc9-8c74-1eee3538d7b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:11.417737 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631c9c02-f565-4fc9-8c74-1eee3538d7b5 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:11.428427 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1230/2459] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:45:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:45:11.725917 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad94a6e-19cf-4795-a980-f2e38715d9f6 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:11.728137 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-cad94a6e-19cf-4795-a980-f2e38715d9f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:11.744845 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad94a6e-19cf-4795-a980-f2e38715d9f6 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:11.762588 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc73aa-8239-4426-b2a6-46b9d8e7a840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:11.763292 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc73aa-8239-4426-b2a6-46b9d8e7a840 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:11.763521 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc73aa-8239-4426-b2a6-46b9d8e7a840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:11.763795 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc73aa-8239-4426-b2a6-46b9d8e7a840 admin admin] PATH_INFO: `/v3/projects/c4417fe411d949ab9eeaa4366ac0be95` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:11.765282 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdc73aa-8239-4426-b2a6-46b9d8e7a840 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4417fe411d949ab9eeaa4366ac0be95)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:11.767115 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdc73aa-8239-4426-b2a6-46b9d8e7a840 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:11.819981 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-1bdc73aa-8239-4426-b2a6-46b9d8e7a840 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4417fe411d949ab9eeaa4366ac0be95', 'request_id': 'req-1bdc73aa-8239-4426-b2a6-46b9d8e7a840'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:11.827925 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-1bdc73aa-8239-4426-b2a6-46b9d8e7a840 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4417fe411d949ab9eeaa4366ac0be95', 'request_id': 'req-1bdc73aa-8239-4426-b2a6-46b9d8e7a840'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:11.834458 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-1bdc73aa-8239-4426-b2a6-46b9d8e7a840 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4417fe411d949ab9eeaa4366ac0be95', 'request_id': 'req-1bdc73aa-8239-4426-b2a6-46b9d8e7a840'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:11.848047 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-1bdc73aa-8239-4426-b2a6-46b9d8e7a840 admin admin] The token cache is being invalidate because project c4417fe411d949ab9eeaa4366ac0be95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:45:11.848430 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-1bdc73aa-8239-4426-b2a6-46b9d8e7a840 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:11.849136 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-1bdc73aa-8239-4426-b2a6-46b9d8e7a840 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:11.853978 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1230/2460] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:45:11 2022] DELETE /identity/v3/projects/c4417fe411d949ab9eeaa4366ac0be95 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:45:11.854907 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631c9c02-f565-4fc9-8c74-1eee3538d7b5 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:11.865717 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631c9c02-f565-4fc9-8c74-1eee3538d7b5 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:11.927240 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b814d6a-1c3c-47d6-8703-4c99ef668b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:11.927872 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b814d6a-1c3c-47d6-8703-4c99ef668b34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:11.928150 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b814d6a-1c3c-47d6-8703-4c99ef668b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:11.928459 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b814d6a-1c3c-47d6-8703-4c99ef668b34 admin admin] PATH_INFO: `/v3/users/1fb10bf394d447b98b2c30a95e6c57d8` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:11.929649 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b814d6a-1c3c-47d6-8703-4c99ef668b34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fb10bf394d447b98b2c30a95e6c57d8)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:11.931560 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b814d6a-1c3c-47d6-8703-4c99ef668b34 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:11.987767 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6b814d6a-1c3c-47d6-8703-4c99ef668b34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fb10bf394d447b98b2c30a95e6c57d8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:11.995992 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6b814d6a-1c3c-47d6-8703-4c99ef668b34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fb10bf394d447b98b2c30a95e6c57d8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:12.010030 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6b814d6a-1c3c-47d6-8703-4c99ef668b34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fb10bf394d447b98b2c30a95e6c57d8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:12.021370 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6b814d6a-1c3c-47d6-8703-4c99ef668b34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fb10bf394d447b98b2c30a95e6c57d8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:12.022003 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6b814d6a-1c3c-47d6-8703-4c99ef668b34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fb10bf394d447b98b2c30a95e6c57d8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:12.030299 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1231/2461] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:45:11 2022] DELETE /identity/v3/users/1fb10bf394d447b98b2c30a95e6c57d8 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:45:12.348666 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc73aa-8239-4426-b2a6-46b9d8e7a840 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:12.350338 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1bdc73aa-8239-4426-b2a6-46b9d8e7a840 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:12.373343 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc73aa-8239-4426-b2a6-46b9d8e7a840 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:12.413943 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:45:12.459162 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344273bf-4567-4e23-9313-8c02957dbd7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:12.459807 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344273bf-4567-4e23-9313-8c02957dbd7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:12.460088 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344273bf-4567-4e23-9313-8c02957dbd7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:12.460360 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344273bf-4567-4e23-9313-8c02957dbd7f admin admin] PATH_INFO: `/v3/projects/e518ce5a5dfa458bad7206db1dc8290b` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:12.461516 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344273bf-4567-4e23-9313-8c02957dbd7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e518ce5a5dfa458bad7206db1dc8290b)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:12.463485 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344273bf-4567-4e23-9313-8c02957dbd7f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:12.526235 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-344273bf-4567-4e23-9313-8c02957dbd7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e518ce5a5dfa458bad7206db1dc8290b', 'request_id': 'req-344273bf-4567-4e23-9313-8c02957dbd7f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:12.533702 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-344273bf-4567-4e23-9313-8c02957dbd7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e518ce5a5dfa458bad7206db1dc8290b', 'request_id': 'req-344273bf-4567-4e23-9313-8c02957dbd7f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:12.541278 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-344273bf-4567-4e23-9313-8c02957dbd7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e518ce5a5dfa458bad7206db1dc8290b', 'request_id': 'req-344273bf-4567-4e23-9313-8c02957dbd7f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:12.555296 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-344273bf-4567-4e23-9313-8c02957dbd7f admin admin] The token cache is being invalidate because project e518ce5a5dfa458bad7206db1dc8290b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:45:12.555570 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-344273bf-4567-4e23-9313-8c02957dbd7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:12.556182 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-344273bf-4567-4e23-9313-8c02957dbd7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:12.557737 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1231/2462] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:45:12 2022] DELETE /identity/v3/projects/e518ce5a5dfa458bad7206db1dc8290b => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:45:12.563144 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb3f3e-6932-4ddf-a167-5cae119bfd83 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:12.563541 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb3f3e-6932-4ddf-a167-5cae119bfd83 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:12.563929 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb3f3e-6932-4ddf-a167-5cae119bfd83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:12.586210 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-d1cb3f3e-6932-4ddf-a167-5cae119bfd83 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:45:12.587447 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d1cb3f3e-6932-4ddf-a167-5cae119bfd83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:12.655128 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:45:12.675925 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1232/2463] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:45:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:45:12.677351 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe4a48-4775-4324-b8b2-2bbd239c0ea1 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:12.677634 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe4a48-4775-4324-b8b2-2bbd239c0ea1 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:12.677905 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe4a48-4775-4324-b8b2-2bbd239c0ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:12.699712 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-6bbe4a48-4775-4324-b8b2-2bbd239c0ea1 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:45:12.701119 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6bbe4a48-4775-4324-b8b2-2bbd239c0ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:12.709694 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1232/2464] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:45:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:45:12.713667 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb3f3e-6932-4ddf-a167-5cae119bfd83 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:12.723916 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb3f3e-6932-4ddf-a167-5cae119bfd83 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:12.728658 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c5bac-ae22-4635-8806-e63840be264c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:12.729266 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c5bac-ae22-4635-8806-e63840be264c admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:12.729542 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c5bac-ae22-4635-8806-e63840be264c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:12.729805 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c5bac-ae22-4635-8806-e63840be264c admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:12.731174 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695c5bac-ae22-4635-8806-e63840be264c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:45:12.731474 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695c5bac-ae22-4635-8806-e63840be264c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:12.733080 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695c5bac-ae22-4635-8806-e63840be264c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:12.748998 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1233/2465] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:45:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:45:12.750653 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe4a48-4775-4324-b8b2-2bbd239c0ea1 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:12.761114 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe4a48-4775-4324-b8b2-2bbd239c0ea1 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:12.766126 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1588e56f-2293-425e-a116-dd2dfa6896ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:12.766719 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588e56f-2293-425e-a116-dd2dfa6896ec admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:12.767028 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588e56f-2293-425e-a116-dd2dfa6896ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:12.767310 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588e56f-2293-425e-a116-dd2dfa6896ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:12.767940 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1588e56f-2293-425e-a116-dd2dfa6896ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:12.769867 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1588e56f-2293-425e-a116-dd2dfa6896ec admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:12.800003 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1233/2466] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:45:12 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:45:12.804275 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c5bac-ae22-4635-8806-e63840be264c admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:12.816267 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c5bac-ae22-4635-8806-e63840be264c admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:12.879616 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30445de9-3f9b-4a67-a008-603303187169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:12.880325 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30445de9-3f9b-4a67-a008-603303187169 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:12.880594 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30445de9-3f9b-4a67-a008-603303187169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:12.880793 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30445de9-3f9b-4a67-a008-603303187169 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:12.881505 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30445de9-3f9b-4a67-a008-603303187169 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:12.883405 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30445de9-3f9b-4a67-a008-603303187169 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:12.908784 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1234/2467] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:45:12 2022] POST /identity/v3/users => generated 290 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:45:12.911740 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1588e56f-2293-425e-a116-dd2dfa6896ec admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:12.923879 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1588e56f-2293-425e-a116-dd2dfa6896ec admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:12.928872 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2269d545-4385-4e1b-8359-86b33ff7acad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:12.929583 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269d545-4385-4e1b-8359-86b33ff7acad admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:12.929965 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269d545-4385-4e1b-8359-86b33ff7acad admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:12.930332 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269d545-4385-4e1b-8359-86b33ff7acad admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:12.931151 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2269d545-4385-4e1b-8359-86b33ff7acad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:12.933261 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2269d545-4385-4e1b-8359-86b33ff7acad admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:12.947681 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1234/2468] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:45:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:45:12.949471 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30445de9-3f9b-4a67-a008-603303187169 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:12.960071 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30445de9-3f9b-4a67-a008-603303187169 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:12.965420 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419722bb-e979-45ba-bd9c-8d8a4ccd05de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:12.966028 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419722bb-e979-45ba-bd9c-8d8a4ccd05de admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:12.966309 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419722bb-e979-45ba-bd9c-8d8a4ccd05de admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:12.966582 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419722bb-e979-45ba-bd9c-8d8a4ccd05de admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:12.967218 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419722bb-e979-45ba-bd9c-8d8a4ccd05de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:12.968915 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419722bb-e979-45ba-bd9c-8d8a4ccd05de admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:12.980203 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1235/2469] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:45:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:45:12.984942 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2269d545-4385-4e1b-8359-86b33ff7acad admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:12.995904 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2269d545-4385-4e1b-8359-86b33ff7acad admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:13.001077 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b6de45-80fe-4b19-995c-95cbaf4522db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:13.001770 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b6de45-80fe-4b19-995c-95cbaf4522db admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:13.002023 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b6de45-80fe-4b19-995c-95cbaf4522db admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:13.002344 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b6de45-80fe-4b19-995c-95cbaf4522db admin admin] PATH_INFO: `/v3/projects/3e925bfd30d94a1cb6bc2158b676c2ea/users/abe51df42ea84f96b77600f5d7ade1a8/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:13.019637 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b6de45-80fe-4b19-995c-95cbaf4522db admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e925bfd30d94a1cb6bc2158b676c2ea, user_id=abe51df42ea84f96b77600f5d7ade1a8, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:13.022716 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b6de45-80fe-4b19-995c-95cbaf4522db admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:13.038824 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1235/2470] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:45:12 2022] PUT /identity/v3/projects/3e925bfd30d94a1cb6bc2158b676c2ea/users/abe51df42ea84f96b77600f5d7ade1a8/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:45:13.049875 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad8d06a-66f1-4afe-8c97-555b72e1aa58 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:13.050172 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad8d06a-66f1-4afe-8c97-555b72e1aa58 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:13.050494 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad8d06a-66f1-4afe-8c97-555b72e1aa58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:13.082546 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-5ad8d06a-66f1-4afe-8c97-555b72e1aa58 None None] MFA Rules not processed for user `abe51df42ea84f96b77600f5d7ade1a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:45:13.084283 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5ad8d06a-66f1-4afe-8c97-555b72e1aa58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:13.159808 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1236/2471] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:45:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:45:13.160840 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b6de45-80fe-4b19-995c-95cbaf4522db admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:13.228855 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:45:13.235763 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b6de45-80fe-4b19-995c-95cbaf4522db admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:13.241497 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54a19c4-d4e5-49f7-bbfa-0bda76a44af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:13.242106 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a19c4-d4e5-49f7-bbfa-0bda76a44af6 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:13.242387 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a19c4-d4e5-49f7-bbfa-0bda76a44af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:13.242657 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a19c4-d4e5-49f7-bbfa-0bda76a44af6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:13.243299 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54a19c4-d4e5-49f7-bbfa-0bda76a44af6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:13.245475 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54a19c4-d4e5-49f7-bbfa-0bda76a44af6 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:13.298556 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1236/2472] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:45:13 2022] POST /identity/v3/projects => generated 379 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:45:13.299923 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad8d06a-66f1-4afe-8c97-555b72e1aa58 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:13.312687 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad8d06a-66f1-4afe-8c97-555b72e1aa58 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:13.376417 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:45:13.387925 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ba61f-0b34-4940-9a61-7523d912d1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:13.388901 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ba61f-0b34-4940-9a61-7523d912d1a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:13.389478 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ba61f-0b34-4940-9a61-7523d912d1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:13.389822 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ba61f-0b34-4940-9a61-7523d912d1a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:13.390519 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9ba61f-0b34-4940-9a61-7523d912d1a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:13.392564 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9ba61f-0b34-4940-9a61-7523d912d1a1 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:13.422746 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54a19c4-d4e5-49f7-bbfa-0bda76a44af6 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:13.424717 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1237/2473] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:45:13 2022] POST /identity/v3/users => generated 296 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:45:13.436281 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54a19c4-d4e5-49f7-bbfa-0bda76a44af6 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:13.441286 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d07841-f47b-40cc-bedc-bc0ed819f83d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:13.441894 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d07841-f47b-40cc-bedc-bc0ed819f83d admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:13.442174 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d07841-f47b-40cc-bedc-bc0ed819f83d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:13.442441 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d07841-f47b-40cc-bedc-bc0ed819f83d admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:13.443117 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d07841-f47b-40cc-bedc-bc0ed819f83d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:13.444740 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d07841-f47b-40cc-bedc-bc0ed819f83d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:13.460570 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1237/2474] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 09:45:13.462276 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ba61f-0b34-4940-9a61-7523d912d1a1 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:13.480114 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ba61f-0b34-4940-9a61-7523d912d1a1 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:13.485661 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56176a47-bcf2-48bc-827b-891e2b22338b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:13.486351 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56176a47-bcf2-48bc-827b-891e2b22338b admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:13.486711 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56176a47-bcf2-48bc-827b-891e2b22338b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:13.486991 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56176a47-bcf2-48bc-827b-891e2b22338b admin admin] PATH_INFO: `/v3/projects/ac99353c1e634f3da68c0f1d4a1830b2/users/225b11da97a847c5bd74d00257eb4125/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:13.504777 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56176a47-bcf2-48bc-827b-891e2b22338b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac99353c1e634f3da68c0f1d4a1830b2, user_id=225b11da97a847c5bd74d00257eb4125, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:13.507543 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56176a47-bcf2-48bc-827b-891e2b22338b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:13.524552 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1238/2475] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:45:13 2022] PUT /identity/v3/projects/ac99353c1e634f3da68c0f1d4a1830b2/users/225b11da97a847c5bd74d00257eb4125/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:45:13.529516 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d07841-f47b-40cc-bedc-bc0ed819f83d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:13.540577 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d07841-f47b-40cc-bedc-bc0ed819f83d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:13.613694 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e3f91b-5c35-42f7-8855-bf25d3750cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:13.614436 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e3f91b-5c35-42f7-8855-bf25d3750cdf admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:13.614717 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e3f91b-5c35-42f7-8855-bf25d3750cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:13.614989 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e3f91b-5c35-42f7-8855-bf25d3750cdf admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:13.615634 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e3f91b-5c35-42f7-8855-bf25d3750cdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:13.617308 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e3f91b-5c35-42f7-8855-bf25d3750cdf admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:13.631560 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1238/2476] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:45:13.635838 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56176a47-bcf2-48bc-827b-891e2b22338b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:13.648717 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56176a47-bcf2-48bc-827b-891e2b22338b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:13.653983 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ddfea-4567-433a-bd83-052045b4ece6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:13.654841 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ddfea-4567-433a-bd83-052045b4ece6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:13.656824 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ddfea-4567-433a-bd83-052045b4ece6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:13.657174 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ddfea-4567-433a-bd83-052045b4ece6 admin admin] PATH_INFO: `/v3/projects/ac99353c1e634f3da68c0f1d4a1830b2/users/225b11da97a847c5bd74d00257eb4125/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:13.659180 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083ddfea-4567-433a-bd83-052045b4ece6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac99353c1e634f3da68c0f1d4a1830b2, user_id=225b11da97a847c5bd74d00257eb4125, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:13.661904 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083ddfea-4567-433a-bd83-052045b4ece6 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:13.672916 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1239/2477] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:45:13 2022] PUT /identity/v3/projects/ac99353c1e634f3da68c0f1d4a1830b2/users/225b11da97a847c5bd74d00257eb4125/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:45:13.684463 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9545bb-1ac5-403b-90c7-d43375e3f2b4 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:13.684897 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9545bb-1ac5-403b-90c7-d43375e3f2b4 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:13.685285 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9545bb-1ac5-403b-90c7-d43375e3f2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:13.721974 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-0a9545bb-1ac5-403b-90c7-d43375e3f2b4 None None] MFA Rules not processed for user `225b11da97a847c5bd74d00257eb4125`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:45:13.723563 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0a9545bb-1ac5-403b-90c7-d43375e3f2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:13.816470 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1239/2478] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:45:13 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:45:13.832612 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ddfea-4567-433a-bd83-052045b4ece6 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:13.834341 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-083ddfea-4567-433a-bd83-052045b4ece6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:13.901223 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ddfea-4567-433a-bd83-052045b4ece6 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:13.905014 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32dd52-90e9-404a-8003-ecb4be1e0056 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:13.905622 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32dd52-90e9-404a-8003-ecb4be1e0056 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:13.905898 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32dd52-90e9-404a-8003-ecb4be1e0056 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:13.906163 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32dd52-90e9-404a-8003-ecb4be1e0056 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:13.915919 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a32dd52-90e9-404a-8003-ecb4be1e0056 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:13.917417 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a32dd52-90e9-404a-8003-ecb4be1e0056 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:13.927884 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1240/2479] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:45:13 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:45:14.316161 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9545bb-1ac5-403b-90c7-d43375e3f2b4 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:14.326036 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9545bb-1ac5-403b-90c7-d43375e3f2b4 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:14.331592 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc43749-14ca-4f87-b0b1-4f3539f7a1ca tempest-VolumeQuotaClassesTest-963151180 tempest-VolumeQuotaClassesTest-963151180-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '225b11da97a847c5bd74d00257eb4125', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac99353c1e634f3da68c0f1d4a1830b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:14.332555 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc43749-14ca-4f87-b0b1-4f3539f7a1ca tempest-VolumeQuotaClassesTest-963151180 tempest-VolumeQuotaClassesTest-963151180-project-admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:14.332555 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc43749-14ca-4f87-b0b1-4f3539f7a1ca tempest-VolumeQuotaClassesTest-963151180 tempest-VolumeQuotaClassesTest-963151180-project-admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:14.333098 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc43749-14ca-4f87-b0b1-4f3539f7a1ca tempest-VolumeQuotaClassesTest-963151180 tempest-VolumeQuotaClassesTest-963151180-project-admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:14.334391 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc43749-14ca-4f87-b0b1-4f3539f7a1ca tempest-VolumeQuotaClassesTest-963151180 tempest-VolumeQuotaClassesTest-963151180-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:45:14.335364 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc43749-14ca-4f87-b0b1-4f3539f7a1ca tempest-VolumeQuotaClassesTest-963151180 tempest-VolumeQuotaClassesTest-963151180-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:14.337268 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc43749-14ca-4f87-b0b1-4f3539f7a1ca tempest-VolumeQuotaClassesTest-963151180 tempest-VolumeQuotaClassesTest-963151180-project-admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:14.351014 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1240/2480] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:45:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:45:14.356368 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32dd52-90e9-404a-8003-ecb4be1e0056 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:14.368625 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32dd52-90e9-404a-8003-ecb4be1e0056 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:14.374903 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8047588-6fcf-4b30-babd-548c9e770b59 tempest-VolumeQuotaClassesTest-963151180 tempest-VolumeQuotaClassesTest-963151180-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '225b11da97a847c5bd74d00257eb4125', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac99353c1e634f3da68c0f1d4a1830b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:14.374903 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8047588-6fcf-4b30-babd-548c9e770b59 tempest-VolumeQuotaClassesTest-963151180 tempest-VolumeQuotaClassesTest-963151180-project-admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:14.374903 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8047588-6fcf-4b30-babd-548c9e770b59 tempest-VolumeQuotaClassesTest-963151180 tempest-VolumeQuotaClassesTest-963151180-project-admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:14.375296 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8047588-6fcf-4b30-babd-548c9e770b59 tempest-VolumeQuotaClassesTest-963151180 tempest-VolumeQuotaClassesTest-963151180-project-admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:14.375296 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8047588-6fcf-4b30-babd-548c9e770b59 tempest-VolumeQuotaClassesTest-963151180 tempest-VolumeQuotaClassesTest-963151180-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:14.377258 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8047588-6fcf-4b30-babd-548c9e770b59 tempest-VolumeQuotaClassesTest-963151180 tempest-VolumeQuotaClassesTest-963151180-project-admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:14.413282 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1241/2481] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:45:14 2022] POST /identity/v3/projects => generated 353 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:45:14.414355 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc43749-14ca-4f87-b0b1-4f3539f7a1ca tempest-VolumeQuotaClassesTest-963151180 tempest-VolumeQuotaClassesTest-963151180-project-admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:14.427819 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc43749-14ca-4f87-b0b1-4f3539f7a1ca tempest-VolumeQuotaClassesTest-963151180 tempest-VolumeQuotaClassesTest-963151180-project-admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:14.474946 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:45:14.510736 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3119306f-b0f3-4abc-a2f8-7571bd9254b2 tempest-VolumeQuotaClassesTest-963151180 tempest-VolumeQuotaClassesTest-963151180-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '225b11da97a847c5bd74d00257eb4125', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac99353c1e634f3da68c0f1d4a1830b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:14.511586 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3119306f-b0f3-4abc-a2f8-7571bd9254b2 tempest-VolumeQuotaClassesTest-963151180 tempest-VolumeQuotaClassesTest-963151180-project-admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:14.512038 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3119306f-b0f3-4abc-a2f8-7571bd9254b2 tempest-VolumeQuotaClassesTest-963151180 tempest-VolumeQuotaClassesTest-963151180-project-admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:14.512454 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3119306f-b0f3-4abc-a2f8-7571bd9254b2 tempest-VolumeQuotaClassesTest-963151180 tempest-VolumeQuotaClassesTest-963151180-project-admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:14.513387 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3119306f-b0f3-4abc-a2f8-7571bd9254b2 tempest-VolumeQuotaClassesTest-963151180 tempest-VolumeQuotaClassesTest-963151180-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:45:14.513651 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3119306f-b0f3-4abc-a2f8-7571bd9254b2 tempest-VolumeQuotaClassesTest-963151180 tempest-VolumeQuotaClassesTest-963151180-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:14.517960 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3119306f-b0f3-4abc-a2f8-7571bd9254b2 tempest-VolumeQuotaClassesTest-963151180 tempest-VolumeQuotaClassesTest-963151180-project-admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:14.535285 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1241/2482] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:45:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:45:14.575657 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8047588-6fcf-4b30-babd-548c9e770b59 tempest-VolumeQuotaClassesTest-963151180 tempest-VolumeQuotaClassesTest-963151180-project-admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:14.588008 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8047588-6fcf-4b30-babd-548c9e770b59 tempest-VolumeQuotaClassesTest-963151180 tempest-VolumeQuotaClassesTest-963151180-project-admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:14.596690 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4537f94c-3c1f-4501-a0ab-bc485e3d56ac tempest-VolumeQuotaClassesTest-963151180 tempest-VolumeQuotaClassesTest-963151180-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '225b11da97a847c5bd74d00257eb4125', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac99353c1e634f3da68c0f1d4a1830b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:14.597499 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4537f94c-3c1f-4501-a0ab-bc485e3d56ac tempest-VolumeQuotaClassesTest-963151180 tempest-VolumeQuotaClassesTest-963151180-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:14.597890 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4537f94c-3c1f-4501-a0ab-bc485e3d56ac tempest-VolumeQuotaClassesTest-963151180 tempest-VolumeQuotaClassesTest-963151180-project-admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:14.598307 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4537f94c-3c1f-4501-a0ab-bc485e3d56ac tempest-VolumeQuotaClassesTest-963151180 tempest-VolumeQuotaClassesTest-963151180-project-admin] PATH_INFO: `/v3/projects/1ed7e0e3253940a5a2203503f49abb32` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:14.599754 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4537f94c-3c1f-4501-a0ab-bc485e3d56ac tempest-VolumeQuotaClassesTest-963151180 tempest-VolumeQuotaClassesTest-963151180-project-admin] RBAC: Authorizing `identity:delete_project(project_id=1ed7e0e3253940a5a2203503f49abb32)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:14.601792 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4537f94c-3c1f-4501-a0ab-bc485e3d56ac tempest-VolumeQuotaClassesTest-963151180 tempest-VolumeQuotaClassesTest-963151180-project-admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:14.664181 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4537f94c-3c1f-4501-a0ab-bc485e3d56ac tempest-VolumeQuotaClassesTest-963151180 tempest-VolumeQuotaClassesTest-963151180-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ed7e0e3253940a5a2203503f49abb32', 'request_id': 'req-4537f94c-3c1f-4501-a0ab-bc485e3d56ac'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:14.673140 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4537f94c-3c1f-4501-a0ab-bc485e3d56ac tempest-VolumeQuotaClassesTest-963151180 tempest-VolumeQuotaClassesTest-963151180-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ed7e0e3253940a5a2203503f49abb32', 'request_id': 'req-4537f94c-3c1f-4501-a0ab-bc485e3d56ac'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:14.686826 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4537f94c-3c1f-4501-a0ab-bc485e3d56ac tempest-VolumeQuotaClassesTest-963151180 tempest-VolumeQuotaClassesTest-963151180-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ed7e0e3253940a5a2203503f49abb32', 'request_id': 'req-4537f94c-3c1f-4501-a0ab-bc485e3d56ac'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:14.710600 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4537f94c-3c1f-4501-a0ab-bc485e3d56ac tempest-VolumeQuotaClassesTest-963151180 tempest-VolumeQuotaClassesTest-963151180-project-admin] The token cache is being invalidate because project 1ed7e0e3253940a5a2203503f49abb32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:45:14.711032 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4537f94c-3c1f-4501-a0ab-bc485e3d56ac tempest-VolumeQuotaClassesTest-963151180 tempest-VolumeQuotaClassesTest-963151180-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:14.711792 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4537f94c-3c1f-4501-a0ab-bc485e3d56ac tempest-VolumeQuotaClassesTest-963151180 tempest-VolumeQuotaClassesTest-963151180-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:14.713616 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1242/2483] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:45:14 2022] DELETE /identity/v3/projects/1ed7e0e3253940a5a2203503f49abb32 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:45:14.928959 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3119306f-b0f3-4abc-a2f8-7571bd9254b2 tempest-VolumeQuotaClassesTest-963151180 tempest-VolumeQuotaClassesTest-963151180-project-admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:14.930759 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3119306f-b0f3-4abc-a2f8-7571bd9254b2 tempest-VolumeQuotaClassesTest-963151180 tempest-VolumeQuotaClassesTest-963151180-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:15.007911 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3119306f-b0f3-4abc-a2f8-7571bd9254b2 tempest-VolumeQuotaClassesTest-963151180 tempest-VolumeQuotaClassesTest-963151180-project-admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:15.071245 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f59fcf7-6bb0-4f3d-ad1f-7df22e243d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:15.072423 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f59fcf7-6bb0-4f3d-ad1f-7df22e243d56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:15.072888 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f59fcf7-6bb0-4f3d-ad1f-7df22e243d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:15.073182 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f59fcf7-6bb0-4f3d-ad1f-7df22e243d56 admin admin] PATH_INFO: `/v3/users/abe51df42ea84f96b77600f5d7ade1a8` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:15.074505 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f59fcf7-6bb0-4f3d-ad1f-7df22e243d56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abe51df42ea84f96b77600f5d7ade1a8)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:15.076689 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f59fcf7-6bb0-4f3d-ad1f-7df22e243d56 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:15.132677 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4f59fcf7-6bb0-4f3d-ad1f-7df22e243d56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abe51df42ea84f96b77600f5d7ade1a8'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:15.140596 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4f59fcf7-6bb0-4f3d-ad1f-7df22e243d56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abe51df42ea84f96b77600f5d7ade1a8'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:15.154853 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4f59fcf7-6bb0-4f3d-ad1f-7df22e243d56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abe51df42ea84f96b77600f5d7ade1a8'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:15.164525 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4f59fcf7-6bb0-4f3d-ad1f-7df22e243d56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abe51df42ea84f96b77600f5d7ade1a8'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:15.165133 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4f59fcf7-6bb0-4f3d-ad1f-7df22e243d56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abe51df42ea84f96b77600f5d7ade1a8'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:15.167627 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:45:15.175839 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1242/2484] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:45:14 2022] DELETE /identity/v3/users/abe51df42ea84f96b77600f5d7ade1a8 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:45:15.180609 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447972e-ea07-4ad8-b70f-964ff7d18c62 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:15.181102 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447972e-ea07-4ad8-b70f-964ff7d18c62 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:15.181387 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447972e-ea07-4ad8-b70f-964ff7d18c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:15.201649 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-e447972e-ea07-4ad8-b70f-964ff7d18c62 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:45:15.203008 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e447972e-ea07-4ad8-b70f-964ff7d18c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:15.208718 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:45:15.280197 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1243/2485] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:45:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:45:15.287757 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f59fcf7-6bb0-4f3d-ad1f-7df22e243d56 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:15.289228 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4f59fcf7-6bb0-4f3d-ad1f-7df22e243d56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:15.355097 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f59fcf7-6bb0-4f3d-ad1f-7df22e243d56 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:15.360093 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf27954-f5ff-4ef8-8301-bb34c1c3252f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:15.360093 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf27954-f5ff-4ef8-8301-bb34c1c3252f service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:15.360367 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf27954-f5ff-4ef8-8301-bb34c1c3252f service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:15.360638 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf27954-f5ff-4ef8-8301-bb34c1c3252f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:15.372542 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf27954-f5ff-4ef8-8301-bb34c1c3252f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:15.374001 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf27954-f5ff-4ef8-8301-bb34c1c3252f service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:15.386207 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1243/2486] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:45:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 09:45:15.695206 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e447972e-ea07-4ad8-b70f-964ff7d18c62 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:15.697194 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e447972e-ea07-4ad8-b70f-964ff7d18c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:15.726403 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e447972e-ea07-4ad8-b70f-964ff7d18c62 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:15.731653 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236877de-e087-43c1-a088-e24931b0108f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:15.732221 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236877de-e087-43c1-a088-e24931b0108f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:15.732500 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236877de-e087-43c1-a088-e24931b0108f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:15.732850 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236877de-e087-43c1-a088-e24931b0108f admin admin] PATH_INFO: `/v3/projects/3e925bfd30d94a1cb6bc2158b676c2ea` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:15.734034 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236877de-e087-43c1-a088-e24931b0108f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e925bfd30d94a1cb6bc2158b676c2ea)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:15.736053 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236877de-e087-43c1-a088-e24931b0108f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:15.792138 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-236877de-e087-43c1-a088-e24931b0108f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e925bfd30d94a1cb6bc2158b676c2ea', 'request_id': 'req-236877de-e087-43c1-a088-e24931b0108f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:15.798379 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-236877de-e087-43c1-a088-e24931b0108f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e925bfd30d94a1cb6bc2158b676c2ea', 'request_id': 'req-236877de-e087-43c1-a088-e24931b0108f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:15.809211 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-236877de-e087-43c1-a088-e24931b0108f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e925bfd30d94a1cb6bc2158b676c2ea', 'request_id': 'req-236877de-e087-43c1-a088-e24931b0108f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:15.821428 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-236877de-e087-43c1-a088-e24931b0108f admin admin] The token cache is being invalidate because project 3e925bfd30d94a1cb6bc2158b676c2ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:45:15.821617 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-236877de-e087-43c1-a088-e24931b0108f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:15.822517 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-236877de-e087-43c1-a088-e24931b0108f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:15.824869 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1244/2487] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:45:15 2022] DELETE /identity/v3/projects/3e925bfd30d94a1cb6bc2158b676c2ea => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:45:15.828570 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf27954-f5ff-4ef8-8301-bb34c1c3252f service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:15.840124 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf27954-f5ff-4ef8-8301-bb34c1c3252f service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:15.914140 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839996aa-eb6d-4a34-b833-6078b3ae4b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:15.914881 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839996aa-eb6d-4a34-b833-6078b3ae4b40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:15.915226 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839996aa-eb6d-4a34-b833-6078b3ae4b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:15.915568 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839996aa-eb6d-4a34-b833-6078b3ae4b40 admin admin] PATH_INFO: `/v3/users/225b11da97a847c5bd74d00257eb4125` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:15.916817 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839996aa-eb6d-4a34-b833-6078b3ae4b40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=225b11da97a847c5bd74d00257eb4125)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:15.918930 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839996aa-eb6d-4a34-b833-6078b3ae4b40 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:15.975080 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-839996aa-eb6d-4a34-b833-6078b3ae4b40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '225b11da97a847c5bd74d00257eb4125'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:15.984479 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-839996aa-eb6d-4a34-b833-6078b3ae4b40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '225b11da97a847c5bd74d00257eb4125'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:15.997921 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-839996aa-eb6d-4a34-b833-6078b3ae4b40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '225b11da97a847c5bd74d00257eb4125'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:16.008501 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-839996aa-eb6d-4a34-b833-6078b3ae4b40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '225b11da97a847c5bd74d00257eb4125'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:16.009471 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-839996aa-eb6d-4a34-b833-6078b3ae4b40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '225b11da97a847c5bd74d00257eb4125'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:16.019737 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1244/2488] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:45:15 2022] DELETE /identity/v3/users/225b11da97a847c5bd74d00257eb4125 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:45:16.333387 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236877de-e087-43c1-a088-e24931b0108f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:16.334803 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-236877de-e087-43c1-a088-e24931b0108f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:16.350507 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236877de-e087-43c1-a088-e24931b0108f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:16.446488 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba9e0e3-36af-40eb-8f2a-c90c76b26030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:16.447309 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9e0e3-36af-40eb-8f2a-c90c76b26030 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:16.447587 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9e0e3-36af-40eb-8f2a-c90c76b26030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:16.447857 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9e0e3-36af-40eb-8f2a-c90c76b26030 admin admin] PATH_INFO: `/v3/projects/ac99353c1e634f3da68c0f1d4a1830b2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:16.451692 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba9e0e3-36af-40eb-8f2a-c90c76b26030 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac99353c1e634f3da68c0f1d4a1830b2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:16.452420 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba9e0e3-36af-40eb-8f2a-c90c76b26030 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:16.504840 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:45:16.513888 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-1ba9e0e3-36af-40eb-8f2a-c90c76b26030 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac99353c1e634f3da68c0f1d4a1830b2', 'request_id': 'req-1ba9e0e3-36af-40eb-8f2a-c90c76b26030'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:16.519364 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-1ba9e0e3-36af-40eb-8f2a-c90c76b26030 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac99353c1e634f3da68c0f1d4a1830b2', 'request_id': 'req-1ba9e0e3-36af-40eb-8f2a-c90c76b26030'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:16.550487 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-1ba9e0e3-36af-40eb-8f2a-c90c76b26030 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac99353c1e634f3da68c0f1d4a1830b2', 'request_id': 'req-1ba9e0e3-36af-40eb-8f2a-c90c76b26030'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:16.567320 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-1ba9e0e3-36af-40eb-8f2a-c90c76b26030 admin admin] The token cache is being invalidate because project ac99353c1e634f3da68c0f1d4a1830b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:45:16.567621 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-1ba9e0e3-36af-40eb-8f2a-c90c76b26030 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:16.568315 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-1ba9e0e3-36af-40eb-8f2a-c90c76b26030 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:16.569880 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1245/2489] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:45:16 2022] DELETE /identity/v3/projects/ac99353c1e634f3da68c0f1d4a1830b2 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:45:16.576118 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf1a06-00f3-43d8-914d-7d2ed86a705b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:16.576403 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf1a06-00f3-43d8-914d-7d2ed86a705b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:16.576711 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf1a06-00f3-43d8-914d-7d2ed86a705b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:16.596264 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-22cf1a06-00f3-43d8-914d-7d2ed86a705b None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:45:16.597528 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-22cf1a06-00f3-43d8-914d-7d2ed86a705b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:16.617417 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:45:16.666053 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1245/2490] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:45:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:45:16.675542 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5369a09-7f1e-4572-b100-da6ff92cb1a2 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:16.675830 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5369a09-7f1e-4572-b100-da6ff92cb1a2 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:16.676091 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5369a09-7f1e-4572-b100-da6ff92cb1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:16.697494 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f5369a09-7f1e-4572-b100-da6ff92cb1a2 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:45:16.698625 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f5369a09-7f1e-4572-b100-da6ff92cb1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:16.707575 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1246/2491] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:45:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:45:16.713658 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cf1a06-00f3-43d8-914d-7d2ed86a705b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:16.724449 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cf1a06-00f3-43d8-914d-7d2ed86a705b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:16.731345 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b3c33-6ed0-4cbc-822b-d12958ab39d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:16.732128 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b3c33-6ed0-4cbc-822b-d12958ab39d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:16.732508 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b3c33-6ed0-4cbc-822b-d12958ab39d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:16.732898 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b3c33-6ed0-4cbc-822b-d12958ab39d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:16.733644 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997b3c33-6ed0-4cbc-822b-d12958ab39d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:45:16.734043 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997b3c33-6ed0-4cbc-822b-d12958ab39d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:16.736262 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997b3c33-6ed0-4cbc-822b-d12958ab39d9 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:16.755252 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1246/2492] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:45:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 09:45:16.756358 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5369a09-7f1e-4572-b100-da6ff92cb1a2 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:16.767174 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5369a09-7f1e-4572-b100-da6ff92cb1a2 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:16.772245 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b42a13-a107-4a1a-9392-409403be8207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:16.772903 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b42a13-a107-4a1a-9392-409403be8207 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:16.773183 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b42a13-a107-4a1a-9392-409403be8207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:16.773458 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b42a13-a107-4a1a-9392-409403be8207 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:16.774514 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b42a13-a107-4a1a-9392-409403be8207 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:16.776098 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b42a13-a107-4a1a-9392-409403be8207 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:16.812369 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1247/2493] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:45:16 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:45:16.813644 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b3c33-6ed0-4cbc-822b-d12958ab39d9 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:16.825238 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b3c33-6ed0-4cbc-822b-d12958ab39d9 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:16.894956 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f199e-98fe-4930-adaf-e4a09c93b2ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:16.895570 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f199e-98fe-4930-adaf-e4a09c93b2ab admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:16.895849 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f199e-98fe-4930-adaf-e4a09c93b2ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:16.896115 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f199e-98fe-4930-adaf-e4a09c93b2ab admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:16.896784 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1f199e-98fe-4930-adaf-e4a09c93b2ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:16.898708 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1f199e-98fe-4930-adaf-e4a09c93b2ab admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:16.926761 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1247/2494] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:45:16 2022] POST /identity/v3/users => generated 287 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:45:16.930861 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b42a13-a107-4a1a-9392-409403be8207 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:16.944659 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b42a13-a107-4a1a-9392-409403be8207 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:16.950495 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbcd6e9-b9fe-415c-9eb4-a922a0b1dda2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:16.953193 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbcd6e9-b9fe-415c-9eb4-a922a0b1dda2 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:16.953586 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbcd6e9-b9fe-415c-9eb4-a922a0b1dda2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:16.953953 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbcd6e9-b9fe-415c-9eb4-a922a0b1dda2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:16.954708 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbcd6e9-b9fe-415c-9eb4-a922a0b1dda2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:16.956395 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbcd6e9-b9fe-415c-9eb4-a922a0b1dda2 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:16.968603 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1248/2495] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:45:16 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:45:16.971822 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f199e-98fe-4930-adaf-e4a09c93b2ab admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:16.982901 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f199e-98fe-4930-adaf-e4a09c93b2ab admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:16.988116 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660784f5-96e3-4bbc-bc57-3a3935a70f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:16.988809 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660784f5-96e3-4bbc-bc57-3a3935a70f2f admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:16.989085 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660784f5-96e3-4bbc-bc57-3a3935a70f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:16.989354 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660784f5-96e3-4bbc-bc57-3a3935a70f2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:16.989995 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660784f5-96e3-4bbc-bc57-3a3935a70f2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:16.991596 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660784f5-96e3-4bbc-bc57-3a3935a70f2f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:17.005288 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1248/2496] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:45:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:45:17.009329 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbcd6e9-b9fe-415c-9eb4-a922a0b1dda2 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:17.020711 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbcd6e9-b9fe-415c-9eb4-a922a0b1dda2 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:17.027923 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df4fd6c-9735-4292-9831-532281801b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:17.028646 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df4fd6c-9735-4292-9831-532281801b8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:17.029027 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df4fd6c-9735-4292-9831-532281801b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:17.029295 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df4fd6c-9735-4292-9831-532281801b8b admin admin] PATH_INFO: `/v3/projects/7f99532b5e9f454fa9d403172e768b9c/users/e2c17efd5165400094f0f127745abc9f/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:17.048700 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df4fd6c-9735-4292-9831-532281801b8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f99532b5e9f454fa9d403172e768b9c, user_id=e2c17efd5165400094f0f127745abc9f, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:17.051658 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df4fd6c-9735-4292-9831-532281801b8b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:17.070249 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1249/2497] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:45:17 2022] PUT /identity/v3/projects/7f99532b5e9f454fa9d403172e768b9c/users/e2c17efd5165400094f0f127745abc9f/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:45:17.075657 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d9d5b-ec17-4c92-b645-5803c61e567b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:17.076196 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d9d5b-ec17-4c92-b645-5803c61e567b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:17.076711 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d9d5b-ec17-4c92-b645-5803c61e567b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:17.101335 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-de6d9d5b-ec17-4c92-b645-5803c61e567b None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:45:17.103215 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-de6d9d5b-ec17-4c92-b645-5803c61e567b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:17.180542 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1249/2498] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:45:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:45:17.193779 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df4fd6c-9735-4292-9831-532281801b8b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:17.196110 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7df4fd6c-9735-4292-9831-532281801b8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:17.264893 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df4fd6c-9735-4292-9831-532281801b8b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:17.273065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f120d3a-d0e7-4e3a-97cd-da9b3ec41276 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:17.273065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f120d3a-d0e7-4e3a-97cd-da9b3ec41276 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:17.273065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f120d3a-d0e7-4e3a-97cd-da9b3ec41276 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:17.273065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f120d3a-d0e7-4e3a-97cd-da9b3ec41276 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:17.285648 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f120d3a-d0e7-4e3a-97cd-da9b3ec41276 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:17.287112 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f120d3a-d0e7-4e3a-97cd-da9b3ec41276 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:17.298190 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1250/2499] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:45:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:45:18.034664 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4f961-abb6-408b-8cc7-56b830b85d6e None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:18.035218 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4f961-abb6-408b-8cc7-56b830b85d6e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:18.036877 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4f961-abb6-408b-8cc7-56b830b85d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:18.084646 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-27e4f961-abb6-408b-8cc7-56b830b85d6e None None] MFA Rules not processed for user `e2c17efd5165400094f0f127745abc9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:45:18.086239 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-27e4f961-abb6-408b-8cc7-56b830b85d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:18.127140 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:45:18.198542 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1250/2500] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:45:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:45:18.201866 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f120d3a-d0e7-4e3a-97cd-da9b3ec41276 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:18.212211 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f120d3a-d0e7-4e3a-97cd-da9b3ec41276 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:18.222198 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31607daf-fc09-4006-ad99-ea73443d1f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:18.222846 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31607daf-fc09-4006-ad99-ea73443d1f5b admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:18.223124 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31607daf-fc09-4006-ad99-ea73443d1f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:18.223400 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31607daf-fc09-4006-ad99-ea73443d1f5b admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:18.224137 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31607daf-fc09-4006-ad99-ea73443d1f5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:18.226465 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31607daf-fc09-4006-ad99-ea73443d1f5b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:18.263660 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1251/2501] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:45:18 2022] POST /identity/v3/projects => generated 371 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:45:18.267669 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e4f961-abb6-408b-8cc7-56b830b85d6e None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:18.286573 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e4f961-abb6-408b-8cc7-56b830b85d6e None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:18.356254 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5217518b-a421-490e-acb0-9fc5526a51f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:18.357073 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217518b-a421-490e-acb0-9fc5526a51f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:18.357370 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217518b-a421-490e-acb0-9fc5526a51f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:18.357672 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217518b-a421-490e-acb0-9fc5526a51f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:18.358267 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5217518b-a421-490e-acb0-9fc5526a51f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:18.360308 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5217518b-a421-490e-acb0-9fc5526a51f7 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:18.391052 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1251/2502] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:45:18 2022] POST /identity/v3/users => generated 292 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:45:18.394791 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31607daf-fc09-4006-ad99-ea73443d1f5b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:18.409489 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31607daf-fc09-4006-ad99-ea73443d1f5b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:18.417796 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd8ae7-d98f-4876-81ac-96a86af1b4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:18.419338 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd8ae7-d98f-4876-81ac-96a86af1b4ba admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:18.419920 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd8ae7-d98f-4876-81ac-96a86af1b4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:18.420287 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd8ae7-d98f-4876-81ac-96a86af1b4ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:18.421727 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd8ae7-d98f-4876-81ac-96a86af1b4ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:18.424243 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd8ae7-d98f-4876-81ac-96a86af1b4ba admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:18.442204 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1252/2503] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:45:18 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:45:18.444202 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5217518b-a421-490e-acb0-9fc5526a51f7 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:18.459482 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5217518b-a421-490e-acb0-9fc5526a51f7 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:18.466904 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c50f60c-c3e4-4341-bd39-24d34fb9f2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:18.467812 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50f60c-c3e4-4341-bd39-24d34fb9f2d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:18.468164 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50f60c-c3e4-4341-bd39-24d34fb9f2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:18.468550 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50f60c-c3e4-4341-bd39-24d34fb9f2d6 admin admin] PATH_INFO: `/v3/projects/01d3dedf43b14d749906a32f4669fa71/users/40632fefbeb845348844f876d07369a4/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:18.488024 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c50f60c-c3e4-4341-bd39-24d34fb9f2d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01d3dedf43b14d749906a32f4669fa71, user_id=40632fefbeb845348844f876d07369a4, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:18.492829 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c50f60c-c3e4-4341-bd39-24d34fb9f2d6 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:18.511009 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1252/2504] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:45:18 2022] PUT /identity/v3/projects/01d3dedf43b14d749906a32f4669fa71/users/40632fefbeb845348844f876d07369a4/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 09:45:18.515057 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd8ae7-d98f-4876-81ac-96a86af1b4ba admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:18.529503 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd8ae7-d98f-4876-81ac-96a86af1b4ba admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:18.571856 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:45:18.616980 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6088ee59-a086-43d9-9c5b-789693f3b8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:18.617678 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088ee59-a086-43d9-9c5b-789693f3b8e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:18.617964 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088ee59-a086-43d9-9c5b-789693f3b8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:18.618236 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088ee59-a086-43d9-9c5b-789693f3b8e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:18.618881 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6088ee59-a086-43d9-9c5b-789693f3b8e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:18.620469 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6088ee59-a086-43d9-9c5b-789693f3b8e4 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:18.636645 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1253/2505] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:45:18 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 09:45:18.638961 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c50f60c-c3e4-4341-bd39-24d34fb9f2d6 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:18.651787 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c50f60c-c3e4-4341-bd39-24d34fb9f2d6 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:18.659494 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b481f4-5443-436e-abfc-abd827fde797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:18.660335 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b481f4-5443-436e-abfc-abd827fde797 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:18.660786 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b481f4-5443-436e-abfc-abd827fde797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:18.661194 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b481f4-5443-436e-abfc-abd827fde797 admin admin] PATH_INFO: `/v3/projects/01d3dedf43b14d749906a32f4669fa71/users/40632fefbeb845348844f876d07369a4/roles/0dfd839b7a194aee96f82f4ebc1620a7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:18.663933 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b481f4-5443-436e-abfc-abd827fde797 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01d3dedf43b14d749906a32f4669fa71, user_id=40632fefbeb845348844f876d07369a4, role_id=0dfd839b7a194aee96f82f4ebc1620a7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:18.667557 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b481f4-5443-436e-abfc-abd827fde797 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:18.683080 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1253/2506] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:45:18 2022] PUT /identity/v3/projects/01d3dedf43b14d749906a32f4669fa71/users/40632fefbeb845348844f876d07369a4/roles/0dfd839b7a194aee96f82f4ebc1620a7 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:45:19.499923 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7cb6f-2d6a-4bcb-9e09-db5c35de5da3 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:19.500335 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7cb6f-2d6a-4bcb-9e09-db5c35de5da3 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:19.500567 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7cb6f-2d6a-4bcb-9e09-db5c35de5da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:19.546773 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-d5a7cb6f-2d6a-4bcb-9e09-db5c35de5da3 None None] MFA Rules not processed for user `40632fefbeb845348844f876d07369a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:45:19.548089 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d5a7cb6f-2d6a-4bcb-9e09-db5c35de5da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:19.634055 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1254/2507] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:45:19 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:45:19.646835 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b481f4-5443-436e-abfc-abd827fde797 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:19.649717 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-82b481f4-5443-436e-abfc-abd827fde797 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:19.736461 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b481f4-5443-436e-abfc-abd827fde797 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:19.741990 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a88b4-043e-45f9-8477-adcf74d7323b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:19.742998 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a88b4-043e-45f9-8477-adcf74d7323b service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:19.743594 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a88b4-043e-45f9-8477-adcf74d7323b service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:19.744027 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a88b4-043e-45f9-8477-adcf74d7323b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:19.779547 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:45:19.832705 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6a88b4-043e-45f9-8477-adcf74d7323b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:19.834311 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6a88b4-043e-45f9-8477-adcf74d7323b service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:19.844825 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1254/2508] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:45:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:45:21.275745 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a7cb6f-2d6a-4bcb-9e09-db5c35de5da3 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:21.277210 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d5a7cb6f-2d6a-4bcb-9e09-db5c35de5da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:21.294902 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a7cb6f-2d6a-4bcb-9e09-db5c35de5da3 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:21.298815 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daeb7f0-cd77-49d1-bca8-d2b5e21adc0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:21.299430 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daeb7f0-cd77-49d1-bca8-d2b5e21adc0f service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:21.299837 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daeb7f0-cd77-49d1-bca8-d2b5e21adc0f service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:21.299974 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daeb7f0-cd77-49d1-bca8-d2b5e21adc0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:21.310578 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daeb7f0-cd77-49d1-bca8-d2b5e21adc0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:21.311999 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daeb7f0-cd77-49d1-bca8-d2b5e21adc0f service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:21.323510 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1255/2509] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:45:21 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:45:27.086681 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f607b5c-2b9c-4182-96cd-38d476c57f6d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:27.087176 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f607b5c-2b9c-4182-96cd-38d476c57f6d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:27.087280 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f607b5c-2b9c-4182-96cd-38d476c57f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:27.105927 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-0f607b5c-2b9c-4182-96cd-38d476c57f6d None None] MFA Rules not processed for user `7f389510a6a044d8a7487561b169544b`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:45:27.107116 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0f607b5c-2b9c-4182-96cd-38d476c57f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:27.161356 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1255/2510] 10.222.0.70 () {58 vars in 967 bytes} [Thu Oct 27 09:45:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:45:27.173630 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daeb7f0-cd77-49d1-bca8-d2b5e21adc0f service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:27.175101 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8daeb7f0-cd77-49d1-bca8-d2b5e21adc0f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:27.243538 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daeb7f0-cd77-49d1-bca8-d2b5e21adc0f service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:27.249051 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8241bc56-4592-4d09-b74a-53d670a90d08 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eecf79ea4700496c89b1a67c614b34d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:27.249659 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241bc56-4592-4d09-b74a-53d670a90d08 service swift] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:27.249937 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241bc56-4592-4d09-b74a-53d670a90d08 service swift] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:27.250208 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241bc56-4592-4d09-b74a-53d670a90d08 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:27.258533 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8241bc56-4592-4d09-b74a-53d670a90d08 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:27.260091 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8241bc56-4592-4d09-b74a-53d670a90d08 service swift] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:27.270075 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1256/2511] 10.222.0.70 () {60 vars in 1384 bytes} [Thu Oct 27 09:45:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 27 09:45:29.162983 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f607b5c-2b9c-4182-96cd-38d476c57f6d None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:29.164546 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0f607b5c-2b9c-4182-96cd-38d476c57f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:29.230952 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f607b5c-2b9c-4182-96cd-38d476c57f6d None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:29.235573 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc829c6-f105-4204-a29c-82aedc350d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:29.236279 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc829c6-f105-4204-a29c-82aedc350d44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:29.236594 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc829c6-f105-4204-a29c-82aedc350d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:29.236865 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc829c6-f105-4204-a29c-82aedc350d44 admin admin] PATH_INFO: `/v3/users/52ac2b5469904e4a8a742de14fcc8508` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:29.237998 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc829c6-f105-4204-a29c-82aedc350d44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52ac2b5469904e4a8a742de14fcc8508)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:29.239927 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc829c6-f105-4204-a29c-82aedc350d44 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:29.292097 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-fbc829c6-f105-4204-a29c-82aedc350d44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52ac2b5469904e4a8a742de14fcc8508'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:29.299415 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-fbc829c6-f105-4204-a29c-82aedc350d44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52ac2b5469904e4a8a742de14fcc8508'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:29.311246 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:45:29.312836 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-fbc829c6-f105-4204-a29c-82aedc350d44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52ac2b5469904e4a8a742de14fcc8508'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:29.320847 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-fbc829c6-f105-4204-a29c-82aedc350d44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52ac2b5469904e4a8a742de14fcc8508'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:29.321721 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-fbc829c6-f105-4204-a29c-82aedc350d44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52ac2b5469904e4a8a742de14fcc8508'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:29.328376 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1256/2512] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:45:29 2022] DELETE /identity/v3/users/52ac2b5469904e4a8a742de14fcc8508 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:45:29.510242 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8241bc56-4592-4d09-b74a-53d670a90d08 service swift] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:29.525706 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8241bc56-4592-4d09-b74a-53d670a90d08 service swift] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:29.555055 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:45:29.586694 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b87150b-48ea-493f-9355-e1315fb02faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:29.587320 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b87150b-48ea-493f-9355-e1315fb02faf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:29.587604 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b87150b-48ea-493f-9355-e1315fb02faf admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:29.587878 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b87150b-48ea-493f-9355-e1315fb02faf admin admin] PATH_INFO: `/v3/projects/f5b8d4a1ce084cc6b3e84ebb7778e5ae` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:29.589024 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b87150b-48ea-493f-9355-e1315fb02faf admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5b8d4a1ce084cc6b3e84ebb7778e5ae)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:29.590954 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b87150b-48ea-493f-9355-e1315fb02faf admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:29.637949 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-1b87150b-48ea-493f-9355-e1315fb02faf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5b8d4a1ce084cc6b3e84ebb7778e5ae', 'request_id': 'req-1b87150b-48ea-493f-9355-e1315fb02faf'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:29.644251 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-1b87150b-48ea-493f-9355-e1315fb02faf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5b8d4a1ce084cc6b3e84ebb7778e5ae', 'request_id': 'req-1b87150b-48ea-493f-9355-e1315fb02faf'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:29.651568 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-1b87150b-48ea-493f-9355-e1315fb02faf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5b8d4a1ce084cc6b3e84ebb7778e5ae', 'request_id': 'req-1b87150b-48ea-493f-9355-e1315fb02faf'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:29.663270 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-1b87150b-48ea-493f-9355-e1315fb02faf admin admin] The token cache is being invalidate because project f5b8d4a1ce084cc6b3e84ebb7778e5ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:45:29.663557 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-1b87150b-48ea-493f-9355-e1315fb02faf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:29.664183 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-1b87150b-48ea-493f-9355-e1315fb02faf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:29.665674 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1257/2513] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:45:29 2022] DELETE /identity/v3/projects/f5b8d4a1ce084cc6b3e84ebb7778e5ae => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:45:29.671149 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247d9d4-e30c-4b2c-ad4d-adc8b63f7b57 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:29.671437 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247d9d4-e30c-4b2c-ad4d-adc8b63f7b57 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:29.671712 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247d9d4-e30c-4b2c-ad4d-adc8b63f7b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:29.691571 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-3247d9d4-e30c-4b2c-ad4d-adc8b63f7b57 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:45:29.692863 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3247d9d4-e30c-4b2c-ad4d-adc8b63f7b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:29.753014 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1257/2514] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:45:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:45:29.760654 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9fc3ce-3f4d-4160-8cb1-9328bb5f169f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:29.760960 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9fc3ce-3f4d-4160-8cb1-9328bb5f169f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:29.761261 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9fc3ce-3f4d-4160-8cb1-9328bb5f169f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:29.781305 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-eb9fc3ce-3f4d-4160-8cb1-9328bb5f169f None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:45:29.782512 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-eb9fc3ce-3f4d-4160-8cb1-9328bb5f169f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:29.790349 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1258/2515] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:45:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:45:29.795156 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3247d9d4-e30c-4b2c-ad4d-adc8b63f7b57 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:29.804899 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3247d9d4-e30c-4b2c-ad4d-adc8b63f7b57 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:29.809585 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3fc03d-4410-46b8-aa23-76b79dedb924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:29.810118 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3fc03d-4410-46b8-aa23-76b79dedb924 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:29.810396 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3fc03d-4410-46b8-aa23-76b79dedb924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:29.810664 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3fc03d-4410-46b8-aa23-76b79dedb924 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:29.811303 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3fc03d-4410-46b8-aa23-76b79dedb924 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:45:29.811604 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3fc03d-4410-46b8-aa23-76b79dedb924 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:29.813234 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3fc03d-4410-46b8-aa23-76b79dedb924 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:29.824928 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1258/2516] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:45:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:45:29.829069 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9fc3ce-3f4d-4160-8cb1-9328bb5f169f None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:29.838091 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9fc3ce-3f4d-4160-8cb1-9328bb5f169f None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:29.843218 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833f4319-7469-4350-bed0-0a3262d23216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:29.843830 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f4319-7469-4350-bed0-0a3262d23216 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:29.844133 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f4319-7469-4350-bed0-0a3262d23216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:29.844403 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f4319-7469-4350-bed0-0a3262d23216 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:29.845071 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833f4319-7469-4350-bed0-0a3262d23216 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:29.846931 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833f4319-7469-4350-bed0-0a3262d23216 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:29.873989 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1259/2517] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:45:29 2022] POST /identity/v3/projects => generated 391 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:45:29.877924 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3fc03d-4410-46b8-aa23-76b79dedb924 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:29.887784 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3fc03d-4410-46b8-aa23-76b79dedb924 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:29.942494 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54da54-abc4-4c0b-bd71-6d2f2c358413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:29.943089 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54da54-abc4-4c0b-bd71-6d2f2c358413 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:29.943365 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54da54-abc4-4c0b-bd71-6d2f2c358413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:29.943637 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54da54-abc4-4c0b-bd71-6d2f2c358413 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:29.944250 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be54da54-abc4-4c0b-bd71-6d2f2c358413 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:29.946263 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be54da54-abc4-4c0b-bd71-6d2f2c358413 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:29.967555 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1259/2518] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:45:29 2022] POST /identity/v3/users => generated 296 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:45:29.971603 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833f4319-7469-4350-bed0-0a3262d23216 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:29.982167 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833f4319-7469-4350-bed0-0a3262d23216 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:29.987059 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61abb6b2-e6ae-4a05-bc9e-2333e91e2032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:29.987710 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61abb6b2-e6ae-4a05-bc9e-2333e91e2032 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:29.988008 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61abb6b2-e6ae-4a05-bc9e-2333e91e2032 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:29.988282 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61abb6b2-e6ae-4a05-bc9e-2333e91e2032 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:29.988955 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61abb6b2-e6ae-4a05-bc9e-2333e91e2032 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:29.990545 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61abb6b2-e6ae-4a05-bc9e-2333e91e2032 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:30.000601 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1260/2519] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:45:30.004296 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54da54-abc4-4c0b-bd71-6d2f2c358413 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:30.014253 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54da54-abc4-4c0b-bd71-6d2f2c358413 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:30.018901 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d56f71b-8297-47ca-a7a5-bf13d9918fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:30.019490 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56f71b-8297-47ca-a7a5-bf13d9918fd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:30.019766 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56f71b-8297-47ca-a7a5-bf13d9918fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:30.020033 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56f71b-8297-47ca-a7a5-bf13d9918fd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:30.020695 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d56f71b-8297-47ca-a7a5-bf13d9918fd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:30.022279 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d56f71b-8297-47ca-a7a5-bf13d9918fd3 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:30.033150 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1260/2520] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:45:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:45:30.036933 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61abb6b2-e6ae-4a05-bc9e-2333e91e2032 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:30.047293 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61abb6b2-e6ae-4a05-bc9e-2333e91e2032 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:30.051937 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1087f7-1a07-46a2-9145-01edad909fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:30.052604 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1087f7-1a07-46a2-9145-01edad909fc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:30.052883 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1087f7-1a07-46a2-9145-01edad909fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:30.053152 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1087f7-1a07-46a2-9145-01edad909fc1 admin admin] PATH_INFO: `/v3/projects/4f3ac7d11a0b4b33a0040fbd42075012/users/226ea72c46364c31a478eff139e3fb1c/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:30.068656 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1087f7-1a07-46a2-9145-01edad909fc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f3ac7d11a0b4b33a0040fbd42075012, user_id=226ea72c46364c31a478eff139e3fb1c, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:30.071211 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1087f7-1a07-46a2-9145-01edad909fc1 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:30.084949 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1261/2521] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:45:30 2022] PUT /identity/v3/projects/4f3ac7d11a0b4b33a0040fbd42075012/users/226ea72c46364c31a478eff139e3fb1c/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:45:30.095425 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0511e573-f127-494b-9cb4-c816eeec4726 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:30.095706 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0511e573-f127-494b-9cb4-c816eeec4726 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:30.096001 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0511e573-f127-494b-9cb4-c816eeec4726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:30.122873 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-0511e573-f127-494b-9cb4-c816eeec4726 None None] MFA Rules not processed for user `226ea72c46364c31a478eff139e3fb1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:45:30.124076 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0511e573-f127-494b-9cb4-c816eeec4726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:30.169009 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:45:30.190977 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1261/2522] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:45:30 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:45:30.202313 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1087f7-1a07-46a2-9145-01edad909fc1 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:30.203973 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ab1087f7-1a07-46a2-9145-01edad909fc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:30.260286 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1087f7-1a07-46a2-9145-01edad909fc1 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:30.264423 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64011b-f224-4345-97c1-fba2e1d83722 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:30.265051 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64011b-f224-4345-97c1-fba2e1d83722 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:30.265326 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64011b-f224-4345-97c1-fba2e1d83722 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:30.265591 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64011b-f224-4345-97c1-fba2e1d83722 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:30.273944 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a64011b-f224-4345-97c1-fba2e1d83722 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:30.275357 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a64011b-f224-4345-97c1-fba2e1d83722 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:30.279486 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:45:30.286588 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1262/2523] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:45:30 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:45:36.651210 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0511e573-f127-494b-9cb4-c816eeec4726 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:36.653231 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0511e573-f127-494b-9cb4-c816eeec4726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:36.720374 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0511e573-f127-494b-9cb4-c816eeec4726 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:36.724936 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cae0c80-3d1a-4396-8e46-7f17cb67fd38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:36.725559 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae0c80-3d1a-4396-8e46-7f17cb67fd38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:36.725837 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae0c80-3d1a-4396-8e46-7f17cb67fd38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:36.726108 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae0c80-3d1a-4396-8e46-7f17cb67fd38 admin admin] PATH_INFO: `/v3/users/e2c17efd5165400094f0f127745abc9f` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:36.727247 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cae0c80-3d1a-4396-8e46-7f17cb67fd38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2c17efd5165400094f0f127745abc9f)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:36.729224 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cae0c80-3d1a-4396-8e46-7f17cb67fd38 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:36.774707 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6cae0c80-3d1a-4396-8e46-7f17cb67fd38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2c17efd5165400094f0f127745abc9f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:36.782067 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6cae0c80-3d1a-4396-8e46-7f17cb67fd38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2c17efd5165400094f0f127745abc9f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:36.794538 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6cae0c80-3d1a-4396-8e46-7f17cb67fd38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2c17efd5165400094f0f127745abc9f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:36.802040 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6cae0c80-3d1a-4396-8e46-7f17cb67fd38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2c17efd5165400094f0f127745abc9f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:36.802796 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6cae0c80-3d1a-4396-8e46-7f17cb67fd38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2c17efd5165400094f0f127745abc9f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:36.808677 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1262/2524] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:45:36 2022] DELETE /identity/v3/users/e2c17efd5165400094f0f127745abc9f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:45:36.979555 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64011b-f224-4345-97c1-fba2e1d83722 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:36.990553 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64011b-f224-4345-97c1-fba2e1d83722 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:37.053471 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c5946-f4b4-41e7-a3df-d6334bbe9aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:37.054096 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c5946-f4b4-41e7-a3df-d6334bbe9aa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:37.054372 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c5946-f4b4-41e7-a3df-d6334bbe9aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:37.054641 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c5946-f4b4-41e7-a3df-d6334bbe9aa3 admin admin] PATH_INFO: `/v3/projects/7f99532b5e9f454fa9d403172e768b9c` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:37.055724 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73c5946-f4b4-41e7-a3df-d6334bbe9aa3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f99532b5e9f454fa9d403172e768b9c)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:37.057708 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73c5946-f4b4-41e7-a3df-d6334bbe9aa3 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:37.106389 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a73c5946-f4b4-41e7-a3df-d6334bbe9aa3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f99532b5e9f454fa9d403172e768b9c', 'request_id': 'req-a73c5946-f4b4-41e7-a3df-d6334bbe9aa3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:37.111842 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a73c5946-f4b4-41e7-a3df-d6334bbe9aa3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f99532b5e9f454fa9d403172e768b9c', 'request_id': 'req-a73c5946-f4b4-41e7-a3df-d6334bbe9aa3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:37.118739 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a73c5946-f4b4-41e7-a3df-d6334bbe9aa3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f99532b5e9f454fa9d403172e768b9c', 'request_id': 'req-a73c5946-f4b4-41e7-a3df-d6334bbe9aa3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:37.130279 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a73c5946-f4b4-41e7-a3df-d6334bbe9aa3 admin admin] The token cache is being invalidate because project 7f99532b5e9f454fa9d403172e768b9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:45:37.130562 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a73c5946-f4b4-41e7-a3df-d6334bbe9aa3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:37.131206 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a73c5946-f4b4-41e7-a3df-d6334bbe9aa3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:37.132717 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1263/2525] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:45:36 2022] DELETE /identity/v3/projects/7f99532b5e9f454fa9d403172e768b9c => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:45:37.136618 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cae0c80-3d1a-4396-8e46-7f17cb67fd38 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:37.138006 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6cae0c80-3d1a-4396-8e46-7f17cb67fd38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:37.150692 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cae0c80-3d1a-4396-8e46-7f17cb67fd38 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:37.209482 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f2ba0-5a36-4d4d-9a34-e218e82ad6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:37.210107 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f2ba0-5a36-4d4d-9a34-e218e82ad6bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:37.210385 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f2ba0-5a36-4d4d-9a34-e218e82ad6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:37.210655 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f2ba0-5a36-4d4d-9a34-e218e82ad6bf admin admin] PATH_INFO: `/v3/users/40632fefbeb845348844f876d07369a4` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:37.211781 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3f2ba0-5a36-4d4d-9a34-e218e82ad6bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=40632fefbeb845348844f876d07369a4)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:37.213739 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3f2ba0-5a36-4d4d-9a34-e218e82ad6bf admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:37.218739 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:45:37.262640 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ab3f2ba0-5a36-4d4d-9a34-e218e82ad6bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40632fefbeb845348844f876d07369a4'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:37.270024 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ab3f2ba0-5a36-4d4d-9a34-e218e82ad6bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40632fefbeb845348844f876d07369a4'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:37.283002 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ab3f2ba0-5a36-4d4d-9a34-e218e82ad6bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40632fefbeb845348844f876d07369a4'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:37.290303 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ab3f2ba0-5a36-4d4d-9a34-e218e82ad6bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40632fefbeb845348844f876d07369a4'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:37.290908 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ab3f2ba0-5a36-4d4d-9a34-e218e82ad6bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40632fefbeb845348844f876d07369a4'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:37.297549 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1263/2526] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:45:37 2022] DELETE /identity/v3/users/40632fefbeb845348844f876d07369a4 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:45:37.470130 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c5946-f4b4-41e7-a3df-d6334bbe9aa3 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:37.471624 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a73c5946-f4b4-41e7-a3df-d6334bbe9aa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:37.484939 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c5946-f4b4-41e7-a3df-d6334bbe9aa3 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:37.542037 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e544b13c-c11f-443c-b9c8-06824e4005e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:37.543291 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544b13c-c11f-443c-b9c8-06824e4005e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:37.543567 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544b13c-c11f-443c-b9c8-06824e4005e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:37.543833 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544b13c-c11f-443c-b9c8-06824e4005e8 admin admin] PATH_INFO: `/v3/projects/01d3dedf43b14d749906a32f4669fa71` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:37.544971 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e544b13c-c11f-443c-b9c8-06824e4005e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01d3dedf43b14d749906a32f4669fa71)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:37.546866 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e544b13c-c11f-443c-b9c8-06824e4005e8 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:37.580539 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:45:37.595486 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e544b13c-c11f-443c-b9c8-06824e4005e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01d3dedf43b14d749906a32f4669fa71', 'request_id': 'req-e544b13c-c11f-443c-b9c8-06824e4005e8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:37.601041 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e544b13c-c11f-443c-b9c8-06824e4005e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01d3dedf43b14d749906a32f4669fa71', 'request_id': 'req-e544b13c-c11f-443c-b9c8-06824e4005e8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:37.608078 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e544b13c-c11f-443c-b9c8-06824e4005e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01d3dedf43b14d749906a32f4669fa71', 'request_id': 'req-e544b13c-c11f-443c-b9c8-06824e4005e8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:37.618876 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e544b13c-c11f-443c-b9c8-06824e4005e8 admin admin] The token cache is being invalidate because project 01d3dedf43b14d749906a32f4669fa71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:45:37.619162 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e544b13c-c11f-443c-b9c8-06824e4005e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:37.619787 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e544b13c-c11f-443c-b9c8-06824e4005e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:37.621342 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1264/2527] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:45:37 2022] DELETE /identity/v3/projects/01d3dedf43b14d749906a32f4669fa71 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:45:37.626203 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a9b5d-980c-453d-95ea-3c015ad0d4a2 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:37.626490 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a9b5d-980c-453d-95ea-3c015ad0d4a2 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:37.626766 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a9b5d-980c-453d-95ea-3c015ad0d4a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:37.646549 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-dc0a9b5d-980c-453d-95ea-3c015ad0d4a2 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:45:37.647681 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-dc0a9b5d-980c-453d-95ea-3c015ad0d4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:37.705716 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1264/2528] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:45:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:45:37.713450 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f460f-a962-4382-ad99-305845894ed1 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:37.713734 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f460f-a962-4382-ad99-305845894ed1 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:37.714002 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f460f-a962-4382-ad99-305845894ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:37.733501 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-d09f460f-a962-4382-ad99-305845894ed1 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:45:37.734680 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d09f460f-a962-4382-ad99-305845894ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:37.742304 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1265/2529] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:45:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:45:37.746347 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a9b5d-980c-453d-95ea-3c015ad0d4a2 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:37.755828 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a9b5d-980c-453d-95ea-3c015ad0d4a2 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:37.760352 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114a0bb8-2082-4f9a-8d58-26bb828a60da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:37.760973 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a0bb8-2082-4f9a-8d58-26bb828a60da admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:37.761251 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a0bb8-2082-4f9a-8d58-26bb828a60da admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:37.762081 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a0bb8-2082-4f9a-8d58-26bb828a60da admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:37.762712 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114a0bb8-2082-4f9a-8d58-26bb828a60da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:45:37.763011 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114a0bb8-2082-4f9a-8d58-26bb828a60da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:37.764636 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114a0bb8-2082-4f9a-8d58-26bb828a60da admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:37.775905 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1265/2530] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:45:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:45:37.780682 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f460f-a962-4382-ad99-305845894ed1 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:37.789877 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f460f-a962-4382-ad99-305845894ed1 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:37.794539 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ce360-4ed3-49d3-95c7-ca9d02a28e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:37.795127 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ce360-4ed3-49d3-95c7-ca9d02a28e91 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:37.795406 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ce360-4ed3-49d3-95c7-ca9d02a28e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:37.795684 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ce360-4ed3-49d3-95c7-ca9d02a28e91 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:37.796314 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ce360-4ed3-49d3-95c7-ca9d02a28e91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:37.798249 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ce360-4ed3-49d3-95c7-ca9d02a28e91 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:37.825869 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1266/2531] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:45:37 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:45:37.829897 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114a0bb8-2082-4f9a-8d58-26bb828a60da admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:37.840488 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114a0bb8-2082-4f9a-8d58-26bb828a60da admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:37.896058 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db665f0-6de6-4ffb-bb91-15af005f5024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:37.896749 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db665f0-6de6-4ffb-bb91-15af005f5024 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:37.896980 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db665f0-6de6-4ffb-bb91-15af005f5024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:37.897252 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db665f0-6de6-4ffb-bb91-15af005f5024 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:37.897883 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db665f0-6de6-4ffb-bb91-15af005f5024 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:37.899795 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db665f0-6de6-4ffb-bb91-15af005f5024 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:37.915973 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:45:37.921787 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1266/2532] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:45:37 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:45:37.925777 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ce360-4ed3-49d3-95c7-ca9d02a28e91 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:37.936503 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ce360-4ed3-49d3-95c7-ca9d02a28e91 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:37.941100 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcd0914-25dc-4cdb-a616-8fd12d4d515d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:37.941690 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd0914-25dc-4cdb-a616-8fd12d4d515d admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:37.941970 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd0914-25dc-4cdb-a616-8fd12d4d515d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:37.942239 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd0914-25dc-4cdb-a616-8fd12d4d515d admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:37.942867 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcd0914-25dc-4cdb-a616-8fd12d4d515d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:37.944440 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcd0914-25dc-4cdb-a616-8fd12d4d515d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:37.955004 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1267/2533] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:45:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:45:37.958732 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db665f0-6de6-4ffb-bb91-15af005f5024 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:37.968424 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db665f0-6de6-4ffb-bb91-15af005f5024 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:37.973035 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b08811-7685-4738-b3a2-2620c52e2bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:37.973617 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b08811-7685-4738-b3a2-2620c52e2bf3 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:37.973900 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b08811-7685-4738-b3a2-2620c52e2bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:37.974163 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b08811-7685-4738-b3a2-2620c52e2bf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:37.974812 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b08811-7685-4738-b3a2-2620c52e2bf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:37.976390 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b08811-7685-4738-b3a2-2620c52e2bf3 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:37.986777 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1267/2534] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:45:37 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:45:37.990757 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcd0914-25dc-4cdb-a616-8fd12d4d515d admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:38.001013 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcd0914-25dc-4cdb-a616-8fd12d4d515d admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:38.005759 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7647eb-5899-40a8-8b9d-8056a96aa95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:38.006433 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7647eb-5899-40a8-8b9d-8056a96aa95b admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:38.006729 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7647eb-5899-40a8-8b9d-8056a96aa95b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:38.006995 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7647eb-5899-40a8-8b9d-8056a96aa95b admin admin] PATH_INFO: `/v3/projects/207286a076564d0daa70074cf8591e75/users/d4a748daaa5343d98a8f5d56b0e6dab0/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:38.022351 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7647eb-5899-40a8-8b9d-8056a96aa95b admin admin] RBAC: Authorizing `identity:create_grant(project_id=207286a076564d0daa70074cf8591e75, user_id=d4a748daaa5343d98a8f5d56b0e6dab0, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:38.025355 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7647eb-5899-40a8-8b9d-8056a96aa95b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:38.038482 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1268/2535] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:45:37 2022] PUT /identity/v3/projects/207286a076564d0daa70074cf8591e75/users/d4a748daaa5343d98a8f5d56b0e6dab0/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:45:38.104324 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad65c8-78c1-4e24-b8c0-f624172a2cd6 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:38.104617 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad65c8-78c1-4e24-b8c0-f624172a2cd6 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:38.104906 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad65c8-78c1-4e24-b8c0-f624172a2cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:38.136402 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-4dad65c8-78c1-4e24-b8c0-f624172a2cd6 None None] MFA Rules not processed for user `d4a748daaa5343d98a8f5d56b0e6dab0`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:45:38.138235 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4dad65c8-78c1-4e24-b8c0-f624172a2cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:38.205687 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1268/2536] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:45:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:45:38.216748 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7647eb-5899-40a8-8b9d-8056a96aa95b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:38.218734 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6d7647eb-5899-40a8-8b9d-8056a96aa95b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:38.296889 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7647eb-5899-40a8-8b9d-8056a96aa95b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:38.301273 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246e5977-a5c3-484c-972c-35100c2127de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:38.301920 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e5977-a5c3-484c-972c-35100c2127de service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:38.302202 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e5977-a5c3-484c-972c-35100c2127de service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:38.302474 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e5977-a5c3-484c-972c-35100c2127de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:38.312765 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246e5977-a5c3-484c-972c-35100c2127de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:38.314216 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246e5977-a5c3-484c-972c-35100c2127de service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:38.323523 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1269/2537] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:45:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:45:38.342076 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dad65c8-78c1-4e24-b8c0-f624172a2cd6 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:38.403756 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dad65c8-78c1-4e24-b8c0-f624172a2cd6 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:38.408222 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82536dd8-6e1e-4a9b-9fa0-838c32a48249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:38.408871 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82536dd8-6e1e-4a9b-9fa0-838c32a48249 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:38.409150 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82536dd8-6e1e-4a9b-9fa0-838c32a48249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:38.409423 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82536dd8-6e1e-4a9b-9fa0-838c32a48249 admin admin] PATH_INFO: `/v3/users/d4a748daaa5343d98a8f5d56b0e6dab0` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:38.410532 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82536dd8-6e1e-4a9b-9fa0-838c32a48249 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4a748daaa5343d98a8f5d56b0e6dab0)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:38.412496 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82536dd8-6e1e-4a9b-9fa0-838c32a48249 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:38.459528 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-82536dd8-6e1e-4a9b-9fa0-838c32a48249 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4a748daaa5343d98a8f5d56b0e6dab0'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:38.466752 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-82536dd8-6e1e-4a9b-9fa0-838c32a48249 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4a748daaa5343d98a8f5d56b0e6dab0'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:38.478429 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-82536dd8-6e1e-4a9b-9fa0-838c32a48249 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4a748daaa5343d98a8f5d56b0e6dab0'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:38.486490 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-82536dd8-6e1e-4a9b-9fa0-838c32a48249 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4a748daaa5343d98a8f5d56b0e6dab0'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:38.487112 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-82536dd8-6e1e-4a9b-9fa0-838c32a48249 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4a748daaa5343d98a8f5d56b0e6dab0'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:38.488511 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:45:38.494812 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1269/2538] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:45:38 2022] DELETE /identity/v3/users/d4a748daaa5343d98a8f5d56b0e6dab0 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:45:38.499868 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945f414-18db-4f98-96f4-e200709bd53d None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:38.500152 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945f414-18db-4f98-96f4-e200709bd53d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:38.500423 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945f414-18db-4f98-96f4-e200709bd53d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:38.519124 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-e945f414-18db-4f98-96f4-e200709bd53d None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:45:38.520312 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e945f414-18db-4f98-96f4-e200709bd53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:38.568891 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:45:38.579654 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1270/2539] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:45:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:45:38.589388 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82536dd8-6e1e-4a9b-9fa0-838c32a48249 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:38.591073 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-82536dd8-6e1e-4a9b-9fa0-838c32a48249 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:38.648766 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82536dd8-6e1e-4a9b-9fa0-838c32a48249 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:38.653072 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2f7985-bf02-4526-bb48-d376e6b19e4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:38.653672 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f7985-bf02-4526-bb48-d376e6b19e4a service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:38.654003 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f7985-bf02-4526-bb48-d376e6b19e4a service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:38.654223 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f7985-bf02-4526-bb48-d376e6b19e4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:38.663278 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2f7985-bf02-4526-bb48-d376e6b19e4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:38.664732 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2f7985-bf02-4526-bb48-d376e6b19e4a service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:38.674220 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1270/2540] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:45:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:45:38.937992 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e945f414-18db-4f98-96f4-e200709bd53d None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:38.939387 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e945f414-18db-4f98-96f4-e200709bd53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:38.955292 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e945f414-18db-4f98-96f4-e200709bd53d None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:38.959983 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8c97ae-f845-41bb-832d-692b63954b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:38.960650 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c97ae-f845-41bb-832d-692b63954b21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:38.960931 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c97ae-f845-41bb-832d-692b63954b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:38.961202 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c97ae-f845-41bb-832d-692b63954b21 admin admin] PATH_INFO: `/v3/projects/207286a076564d0daa70074cf8591e75` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:38.962323 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8c97ae-f845-41bb-832d-692b63954b21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=207286a076564d0daa70074cf8591e75)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:38.964230 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8c97ae-f845-41bb-832d-692b63954b21 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:39.012179 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3c8c97ae-f845-41bb-832d-692b63954b21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '207286a076564d0daa70074cf8591e75', 'request_id': 'req-3c8c97ae-f845-41bb-832d-692b63954b21'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:39.017655 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3c8c97ae-f845-41bb-832d-692b63954b21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '207286a076564d0daa70074cf8591e75', 'request_id': 'req-3c8c97ae-f845-41bb-832d-692b63954b21'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:39.024792 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3c8c97ae-f845-41bb-832d-692b63954b21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '207286a076564d0daa70074cf8591e75', 'request_id': 'req-3c8c97ae-f845-41bb-832d-692b63954b21'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:39.035717 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3c8c97ae-f845-41bb-832d-692b63954b21 admin admin] The token cache is being invalidate because project 207286a076564d0daa70074cf8591e75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:45:39.035997 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3c8c97ae-f845-41bb-832d-692b63954b21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:39.036646 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3c8c97ae-f845-41bb-832d-692b63954b21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:39.038087 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1271/2541] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:45:38 2022] DELETE /identity/v3/projects/207286a076564d0daa70074cf8591e75 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:45:39.043492 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695562a9-6b2f-4190-b89e-56ab85542d91 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:39.043984 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695562a9-6b2f-4190-b89e-56ab85542d91 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:39.044085 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695562a9-6b2f-4190-b89e-56ab85542d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:39.063857 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-695562a9-6b2f-4190-b89e-56ab85542d91 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:45:39.065069 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-695562a9-6b2f-4190-b89e-56ab85542d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:39.123403 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1271/2542] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:45:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:45:39.130789 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0696a639-34ad-40ec-8f7c-91519eb0af1a None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:39.131072 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0696a639-34ad-40ec-8f7c-91519eb0af1a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:39.131341 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0696a639-34ad-40ec-8f7c-91519eb0af1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:39.150500 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-0696a639-34ad-40ec-8f7c-91519eb0af1a None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:45:39.151678 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0696a639-34ad-40ec-8f7c-91519eb0af1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:39.159735 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1272/2543] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:45:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 09:45:39.163839 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695562a9-6b2f-4190-b89e-56ab85542d91 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:39.174015 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695562a9-6b2f-4190-b89e-56ab85542d91 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:39.178555 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae790afd-022d-41f1-8951-9cea7694f7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:39.179145 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae790afd-022d-41f1-8951-9cea7694f7d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:39.179425 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae790afd-022d-41f1-8951-9cea7694f7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:39.179693 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae790afd-022d-41f1-8951-9cea7694f7d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:39.180386 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae790afd-022d-41f1-8951-9cea7694f7d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:45:39.180662 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae790afd-022d-41f1-8951-9cea7694f7d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:39.182244 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae790afd-022d-41f1-8951-9cea7694f7d8 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:39.194284 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1272/2544] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:45:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:45:39.198625 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0696a639-34ad-40ec-8f7c-91519eb0af1a None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:39.208882 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0696a639-34ad-40ec-8f7c-91519eb0af1a None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:39.213579 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7018153-544a-4f11-ac90-28902bd8f997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:39.214194 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7018153-544a-4f11-ac90-28902bd8f997 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:39.214474 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7018153-544a-4f11-ac90-28902bd8f997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:39.214740 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7018153-544a-4f11-ac90-28902bd8f997 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:39.215366 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7018153-544a-4f11-ac90-28902bd8f997 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:39.217326 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7018153-544a-4f11-ac90-28902bd8f997 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:39.244013 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1273/2545] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:45:39 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 09:45:39.248951 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae790afd-022d-41f1-8951-9cea7694f7d8 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:39.258838 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae790afd-022d-41f1-8951-9cea7694f7d8 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:39.315127 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6797b90a-6cf7-4e25-a17e-9550cc374ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:39.315750 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6797b90a-6cf7-4e25-a17e-9550cc374ec4 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:39.316018 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6797b90a-6cf7-4e25-a17e-9550cc374ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:39.316289 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6797b90a-6cf7-4e25-a17e-9550cc374ec4 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:39.317126 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6797b90a-6cf7-4e25-a17e-9550cc374ec4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:39.318859 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6797b90a-6cf7-4e25-a17e-9550cc374ec4 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:39.344856 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1273/2546] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:45:39 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:45:39.346278 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7018153-544a-4f11-ac90-28902bd8f997 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:39.357145 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7018153-544a-4f11-ac90-28902bd8f997 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:39.362827 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9241eafe-d026-4861-a96e-ba365bad2976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:39.363452 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241eafe-d026-4861-a96e-ba365bad2976 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:39.363732 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241eafe-d026-4861-a96e-ba365bad2976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:39.364020 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241eafe-d026-4861-a96e-ba365bad2976 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:39.364676 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9241eafe-d026-4861-a96e-ba365bad2976 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:39.366256 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9241eafe-d026-4861-a96e-ba365bad2976 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:39.376835 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1274/2547] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:45:39.380925 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6797b90a-6cf7-4e25-a17e-9550cc374ec4 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:39.390929 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6797b90a-6cf7-4e25-a17e-9550cc374ec4 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:39.395576 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef21fa-a51b-4d92-84c9-b5e6a0af2dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:39.396171 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef21fa-a51b-4d92-84c9-b5e6a0af2dfb admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:39.396449 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef21fa-a51b-4d92-84c9-b5e6a0af2dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:39.396754 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef21fa-a51b-4d92-84c9-b5e6a0af2dfb admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:39.397386 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ef21fa-a51b-4d92-84c9-b5e6a0af2dfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:39.398969 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ef21fa-a51b-4d92-84c9-b5e6a0af2dfb admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:39.409460 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1274/2548] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:45:39.413354 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9241eafe-d026-4861-a96e-ba365bad2976 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:39.423853 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9241eafe-d026-4861-a96e-ba365bad2976 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:39.428623 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ed6ea8-5c09-4c90-9029-fa12af5820de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:39.429265 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed6ea8-5c09-4c90-9029-fa12af5820de admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:39.429541 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed6ea8-5c09-4c90-9029-fa12af5820de admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:39.429840 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed6ea8-5c09-4c90-9029-fa12af5820de admin admin] PATH_INFO: `/v3/projects/eab2a4fd1e5b4c22b0f30abaf850a74b/users/b2137313d3174b7a8754e147569f3fe3/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:39.445225 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ed6ea8-5c09-4c90-9029-fa12af5820de admin admin] RBAC: Authorizing `identity:create_grant(project_id=eab2a4fd1e5b4c22b0f30abaf850a74b, user_id=b2137313d3174b7a8754e147569f3fe3, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:39.448103 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ed6ea8-5c09-4c90-9029-fa12af5820de admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:39.461004 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1275/2549] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:45:39 2022] PUT /identity/v3/projects/eab2a4fd1e5b4c22b0f30abaf850a74b/users/b2137313d3174b7a8754e147569f3fe3/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:45:39.472040 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd1951-245e-47fb-b62c-92e2b04705a7 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:39.472319 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd1951-245e-47fb-b62c-92e2b04705a7 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:39.472607 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd1951-245e-47fb-b62c-92e2b04705a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:39.486605 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:45:39.501229 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-5bbd1951-245e-47fb-b62c-92e2b04705a7 None None] MFA Rules not processed for user `b2137313d3174b7a8754e147569f3fe3`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:45:39.502419 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5bbd1951-245e-47fb-b62c-92e2b04705a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:39.564536 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1275/2550] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:45:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:45:39.575970 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ed6ea8-5c09-4c90-9029-fa12af5820de admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:39.577690 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-76ed6ea8-5c09-4c90-9029-fa12af5820de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:39.633729 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ed6ea8-5c09-4c90-9029-fa12af5820de admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:39.637480 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4025a812-0317-4f9d-881e-c1bf1816adc2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72b0538f0e42446887b1d95b2fb41fb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:39.638079 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4025a812-0317-4f9d-881e-c1bf1816adc2 service glance] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:39.638356 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4025a812-0317-4f9d-881e-c1bf1816adc2 service glance] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:39.638626 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4025a812-0317-4f9d-881e-c1bf1816adc2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:39.646907 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4025a812-0317-4f9d-881e-c1bf1816adc2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:39.648320 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4025a812-0317-4f9d-881e-c1bf1816adc2 service glance] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:39.657613 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1276/2551] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:45:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:45:46.704534 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbd1951-245e-47fb-b62c-92e2b04705a7 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:46.706099 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5bbd1951-245e-47fb-b62c-92e2b04705a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:46.770492 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbd1951-245e-47fb-b62c-92e2b04705a7 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:46.775017 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4f91ad-fe68-4ccb-b438-9a9d3f156769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:46.775643 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f91ad-fe68-4ccb-b438-9a9d3f156769 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:46.775923 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f91ad-fe68-4ccb-b438-9a9d3f156769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:46.776194 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f91ad-fe68-4ccb-b438-9a9d3f156769 admin admin] PATH_INFO: `/v3/users/226ea72c46364c31a478eff139e3fb1c` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:46.777471 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4f91ad-fe68-4ccb-b438-9a9d3f156769 admin admin] RBAC: Authorizing `identity:delete_user(user_id=226ea72c46364c31a478eff139e3fb1c)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:46.779497 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4f91ad-fe68-4ccb-b438-9a9d3f156769 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:46.827653 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9b4f91ad-fe68-4ccb-b438-9a9d3f156769 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '226ea72c46364c31a478eff139e3fb1c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:46.834788 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9b4f91ad-fe68-4ccb-b438-9a9d3f156769 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '226ea72c46364c31a478eff139e3fb1c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:46.846780 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9b4f91ad-fe68-4ccb-b438-9a9d3f156769 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '226ea72c46364c31a478eff139e3fb1c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:46.855718 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9b4f91ad-fe68-4ccb-b438-9a9d3f156769 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '226ea72c46364c31a478eff139e3fb1c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:46.856319 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9b4f91ad-fe68-4ccb-b438-9a9d3f156769 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '226ea72c46364c31a478eff139e3fb1c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:46.862444 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1276/2552] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:45:46 2022] DELETE /identity/v3/users/226ea72c46364c31a478eff139e3fb1c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:45:46.866973 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6859048e-6ee2-4196-9309-ebd9e2cf1c92 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:46.867496 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6859048e-6ee2-4196-9309-ebd9e2cf1c92 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:46.867600 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6859048e-6ee2-4196-9309-ebd9e2cf1c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:46.887847 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-6859048e-6ee2-4196-9309-ebd9e2cf1c92 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:45:46.889725 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6859048e-6ee2-4196-9309-ebd9e2cf1c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:46.902137 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:45:46.951268 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1277/2553] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:45:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:45:46.960889 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4f91ad-fe68-4ccb-b438-9a9d3f156769 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:46.962273 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9b4f91ad-fe68-4ccb-b438-9a9d3f156769 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:46.981342 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:45:47.022488 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4f91ad-fe68-4ccb-b438-9a9d3f156769 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:47.026107 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c2cc72-2276-4dba-8c99-25862af72c71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:47.026731 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2cc72-2276-4dba-8c99-25862af72c71 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:47.027002 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2cc72-2276-4dba-8c99-25862af72c71 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:47.027266 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2cc72-2276-4dba-8c99-25862af72c71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:47.036942 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c2cc72-2276-4dba-8c99-25862af72c71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:47.038419 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c2cc72-2276-4dba-8c99-25862af72c71 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:47.048344 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1277/2554] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:45:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:45:47.323146 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6859048e-6ee2-4196-9309-ebd9e2cf1c92 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:47.324618 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6859048e-6ee2-4196-9309-ebd9e2cf1c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:47.336380 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6859048e-6ee2-4196-9309-ebd9e2cf1c92 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:47.341095 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb20dd0-cc25-4c78-bd3d-6d4827f33e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:47.342483 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb20dd0-cc25-4c78-bd3d-6d4827f33e7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:47.342777 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb20dd0-cc25-4c78-bd3d-6d4827f33e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:47.343198 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb20dd0-cc25-4c78-bd3d-6d4827f33e7d admin admin] PATH_INFO: `/v3/projects/4f3ac7d11a0b4b33a0040fbd42075012` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:47.344450 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb20dd0-cc25-4c78-bd3d-6d4827f33e7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f3ac7d11a0b4b33a0040fbd42075012)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:47.348431 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb20dd0-cc25-4c78-bd3d-6d4827f33e7d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:47.395752 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-bdb20dd0-cc25-4c78-bd3d-6d4827f33e7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f3ac7d11a0b4b33a0040fbd42075012', 'request_id': 'req-bdb20dd0-cc25-4c78-bd3d-6d4827f33e7d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:47.401576 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-bdb20dd0-cc25-4c78-bd3d-6d4827f33e7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f3ac7d11a0b4b33a0040fbd42075012', 'request_id': 'req-bdb20dd0-cc25-4c78-bd3d-6d4827f33e7d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:47.410793 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-bdb20dd0-cc25-4c78-bd3d-6d4827f33e7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f3ac7d11a0b4b33a0040fbd42075012', 'request_id': 'req-bdb20dd0-cc25-4c78-bd3d-6d4827f33e7d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:47.424035 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-bdb20dd0-cc25-4c78-bd3d-6d4827f33e7d admin admin] The token cache is being invalidate because project 4f3ac7d11a0b4b33a0040fbd42075012 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:45:47.424686 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-bdb20dd0-cc25-4c78-bd3d-6d4827f33e7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:47.424983 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-bdb20dd0-cc25-4c78-bd3d-6d4827f33e7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:47.426485 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1278/2555] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:45:47 2022] DELETE /identity/v3/projects/4f3ac7d11a0b4b33a0040fbd42075012 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:45:57.043606 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=19 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:45:57.272403 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c2cc72-2276-4dba-8c99-25862af72c71 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:57.273856 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-15c2cc72-2276-4dba-8c99-25862af72c71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:57.287325 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c2cc72-2276-4dba-8c99-25862af72c71 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:57.344096 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8fa15a-72dc-42fc-b3e2-e95cf2a67a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:57.344752 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8fa15a-72dc-42fc-b3e2-e95cf2a67a2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:57.345030 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8fa15a-72dc-42fc-b3e2-e95cf2a67a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:57.345296 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8fa15a-72dc-42fc-b3e2-e95cf2a67a2d admin admin] PATH_INFO: `/v3/users/b2137313d3174b7a8754e147569f3fe3` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:57.346408 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8fa15a-72dc-42fc-b3e2-e95cf2a67a2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2137313d3174b7a8754e147569f3fe3)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:57.348315 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8fa15a-72dc-42fc-b3e2-e95cf2a67a2d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:57.395152 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6d8fa15a-72dc-42fc-b3e2-e95cf2a67a2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2137313d3174b7a8754e147569f3fe3'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:57.402108 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6d8fa15a-72dc-42fc-b3e2-e95cf2a67a2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2137313d3174b7a8754e147569f3fe3'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:57.414189 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6d8fa15a-72dc-42fc-b3e2-e95cf2a67a2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2137313d3174b7a8754e147569f3fe3'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:57.419185 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=45,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:45:57.421436 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6d8fa15a-72dc-42fc-b3e2-e95cf2a67a2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2137313d3174b7a8754e147569f3fe3'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:57.422211 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6d8fa15a-72dc-42fc-b3e2-e95cf2a67a2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2137313d3174b7a8754e147569f3fe3'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:57.428224 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1278/2556] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:45:57 2022] DELETE /identity/v3/users/b2137313d3174b7a8754e147569f3fe3 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:45:57.432738 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa2026-29a8-4dc1-b0bd-7724140b8100 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:57.433026 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa2026-29a8-4dc1-b0bd-7724140b8100 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:57.433302 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa2026-29a8-4dc1-b0bd-7724140b8100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:57.452616 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-c0aa2026-29a8-4dc1-b0bd-7724140b8100 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:45:57.453805 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c0aa2026-29a8-4dc1-b0bd-7724140b8100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:57.513014 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1279/2557] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:45:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:45:57.523174 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8fa15a-72dc-42fc-b3e2-e95cf2a67a2d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:57.524569 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6d8fa15a-72dc-42fc-b3e2-e95cf2a67a2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:57.581522 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8fa15a-72dc-42fc-b3e2-e95cf2a67a2d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:57.585168 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d393f440-197b-4e23-9615-1098b751816b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:57.585756 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393f440-197b-4e23-9615-1098b751816b service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:57.586030 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393f440-197b-4e23-9615-1098b751816b service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:57.586293 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393f440-197b-4e23-9615-1098b751816b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:57.595936 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d393f440-197b-4e23-9615-1098b751816b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:57.597482 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d393f440-197b-4e23-9615-1098b751816b service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:57.606837 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1279/2558] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:45:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:45:57.866032 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aa2026-29a8-4dc1-b0bd-7724140b8100 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:57.867451 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c0aa2026-29a8-4dc1-b0bd-7724140b8100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:57.879876 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aa2026-29a8-4dc1-b0bd-7724140b8100 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:57.885064 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed0044-0834-4a26-8053-6b9e98518c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:57.885617 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed0044-0834-4a26-8053-6b9e98518c98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:57.885894 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed0044-0834-4a26-8053-6b9e98518c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:57.886168 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed0044-0834-4a26-8053-6b9e98518c98 admin admin] PATH_INFO: `/v3/projects/eab2a4fd1e5b4c22b0f30abaf850a74b` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:57.887307 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ed0044-0834-4a26-8053-6b9e98518c98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eab2a4fd1e5b4c22b0f30abaf850a74b)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:57.889923 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ed0044-0834-4a26-8053-6b9e98518c98 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:57.935555 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-92ed0044-0834-4a26-8053-6b9e98518c98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eab2a4fd1e5b4c22b0f30abaf850a74b', 'request_id': 'req-92ed0044-0834-4a26-8053-6b9e98518c98'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:57.942037 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-92ed0044-0834-4a26-8053-6b9e98518c98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eab2a4fd1e5b4c22b0f30abaf850a74b', 'request_id': 'req-92ed0044-0834-4a26-8053-6b9e98518c98'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:57.949686 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-92ed0044-0834-4a26-8053-6b9e98518c98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eab2a4fd1e5b4c22b0f30abaf850a74b', 'request_id': 'req-92ed0044-0834-4a26-8053-6b9e98518c98'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:57.961203 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-92ed0044-0834-4a26-8053-6b9e98518c98 admin admin] The token cache is being invalidate because project eab2a4fd1e5b4c22b0f30abaf850a74b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:45:57.961489 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-92ed0044-0834-4a26-8053-6b9e98518c98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:57.962104 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-92ed0044-0834-4a26-8053-6b9e98518c98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:45:57.963618 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1280/2559] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:45:57 2022] DELETE /identity/v3/projects/eab2a4fd1e5b4c22b0f30abaf850a74b => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:45:57.968427 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ada8fbe-3c31-4304-a9a8-c80b380a7b55 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:57.968745 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ada8fbe-3c31-4304-a9a8-c80b380a7b55 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:57.969014 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ada8fbe-3c31-4304-a9a8-c80b380a7b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:57.987789 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-2ada8fbe-3c31-4304-a9a8-c80b380a7b55 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:45:57.989008 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2ada8fbe-3c31-4304-a9a8-c80b380a7b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:58.006520 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:45:58.049040 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1280/2560] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:45:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:45:58.057850 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aab1d7-8ca5-4f3d-99c8-c433844be72c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:58.058141 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aab1d7-8ca5-4f3d-99c8-c433844be72c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:58.058416 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aab1d7-8ca5-4f3d-99c8-c433844be72c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:58.079232 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-37aab1d7-8ca5-4f3d-99c8-c433844be72c None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:45:58.080436 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-37aab1d7-8ca5-4f3d-99c8-c433844be72c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:58.088289 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1281/2561] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:45:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:45:58.092413 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ada8fbe-3c31-4304-a9a8-c80b380a7b55 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:58.102151 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ada8fbe-3c31-4304-a9a8-c80b380a7b55 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:58.106833 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ada03b-f2cb-4aa3-867e-518ddafa86db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:58.107426 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ada03b-f2cb-4aa3-867e-518ddafa86db admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:58.107730 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ada03b-f2cb-4aa3-867e-518ddafa86db admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:58.108732 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ada03b-f2cb-4aa3-867e-518ddafa86db admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:58.109317 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ada03b-f2cb-4aa3-867e-518ddafa86db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:45:58.109616 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ada03b-f2cb-4aa3-867e-518ddafa86db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:58.111229 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ada03b-f2cb-4aa3-867e-518ddafa86db admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:58.124155 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1281/2562] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:45:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:45:58.128059 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aab1d7-8ca5-4f3d-99c8-c433844be72c None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:58.138667 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aab1d7-8ca5-4f3d-99c8-c433844be72c None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:58.143610 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6491db3-bc33-4e5a-8451-af5ce8959589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:58.144209 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6491db3-bc33-4e5a-8451-af5ce8959589 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:58.144493 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6491db3-bc33-4e5a-8451-af5ce8959589 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:58.144801 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6491db3-bc33-4e5a-8451-af5ce8959589 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:58.145436 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6491db3-bc33-4e5a-8451-af5ce8959589 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:58.147358 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6491db3-bc33-4e5a-8451-af5ce8959589 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:58.176169 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1282/2563] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:45:58 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:45:58.180436 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ada03b-f2cb-4aa3-867e-518ddafa86db admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:58.191097 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ada03b-f2cb-4aa3-867e-518ddafa86db admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:58.245804 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6236d171-2657-4da6-a5ec-4db21be36bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:58.246393 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236d171-2657-4da6-a5ec-4db21be36bad admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:58.246669 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236d171-2657-4da6-a5ec-4db21be36bad admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:58.246932 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236d171-2657-4da6-a5ec-4db21be36bad admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:58.247536 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6236d171-2657-4da6-a5ec-4db21be36bad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:58.250126 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6236d171-2657-4da6-a5ec-4db21be36bad admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:58.270360 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1282/2564] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:45:58 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:45:58.274408 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6491db3-bc33-4e5a-8451-af5ce8959589 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:58.284961 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6491db3-bc33-4e5a-8451-af5ce8959589 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:58.289617 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c5c2fa-d23d-47d6-97d8-e4ed77d931cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:58.290295 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c5c2fa-d23d-47d6-97d8-e4ed77d931cc admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:58.290573 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c5c2fa-d23d-47d6-97d8-e4ed77d931cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:58.290846 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c5c2fa-d23d-47d6-97d8-e4ed77d931cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:58.291479 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c5c2fa-d23d-47d6-97d8-e4ed77d931cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:58.293100 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c5c2fa-d23d-47d6-97d8-e4ed77d931cc admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:58.303876 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1283/2565] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:45:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:45:58.307876 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6236d171-2657-4da6-a5ec-4db21be36bad admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:58.318370 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6236d171-2657-4da6-a5ec-4db21be36bad admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:58.323257 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ada9f-567e-4594-9594-38e20501a8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:58.323849 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ada9f-567e-4594-9594-38e20501a8ec admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:58.324126 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ada9f-567e-4594-9594-38e20501a8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:58.324401 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ada9f-567e-4594-9594-38e20501a8ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:58.325065 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85ada9f-567e-4594-9594-38e20501a8ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:58.326644 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85ada9f-567e-4594-9594-38e20501a8ec admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:58.337097 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1283/2566] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:45:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:45:58.340962 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c5c2fa-d23d-47d6-97d8-e4ed77d931cc admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:58.350956 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c5c2fa-d23d-47d6-97d8-e4ed77d931cc admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:58.355551 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e5459-5882-45db-ab06-0fb6f3831100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:58.356184 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e5459-5882-45db-ab06-0fb6f3831100 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:58.356459 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e5459-5882-45db-ab06-0fb6f3831100 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:58.356764 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e5459-5882-45db-ab06-0fb6f3831100 admin admin] PATH_INFO: `/v3/projects/6cc3a300905949c69fed9679edb69e6c/users/d81521fd864f4692bb3bab6ba7dfc8b9/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:58.371522 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e5459-5882-45db-ab06-0fb6f3831100 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cc3a300905949c69fed9679edb69e6c, user_id=d81521fd864f4692bb3bab6ba7dfc8b9, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:58.374775 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e5459-5882-45db-ab06-0fb6f3831100 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:58.387347 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1284/2567] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:45:58 2022] PUT /identity/v3/projects/6cc3a300905949c69fed9679edb69e6c/users/d81521fd864f4692bb3bab6ba7dfc8b9/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:45:58.397809 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8847ce-f4bd-47f7-8fd2-ae8700e219a9 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:58.398092 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8847ce-f4bd-47f7-8fd2-ae8700e219a9 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:58.398361 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8847ce-f4bd-47f7-8fd2-ae8700e219a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:58.425804 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-9a8847ce-f4bd-47f7-8fd2-ae8700e219a9 None None] MFA Rules not processed for user `d81521fd864f4692bb3bab6ba7dfc8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:45:58.426997 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9a8847ce-f4bd-47f7-8fd2-ae8700e219a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:58.488755 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1284/2568] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:45:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:45:58.500872 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e5459-5882-45db-ab06-0fb6f3831100 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:45:58.502696 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-df2e5459-5882-45db-ab06-0fb6f3831100 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:45:58.525925 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:45:58.562910 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e5459-5882-45db-ab06-0fb6f3831100 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:45:58.566549 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812f2cb7-bda6-4668-9556-3071437d6442 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:45:58.567148 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f2cb7-bda6-4668-9556-3071437d6442 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:45:58.567427 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f2cb7-bda6-4668-9556-3071437d6442 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:45:58.567699 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f2cb7-bda6-4668-9556-3071437d6442 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:45:58.576110 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812f2cb7-bda6-4668-9556-3071437d6442 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:45:58.577620 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812f2cb7-bda6-4668-9556-3071437d6442 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:45:58.586657 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1285/2569] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:45:58 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:46:01.129506 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8847ce-f4bd-47f7-8fd2-ae8700e219a9 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:46:01.147965 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:46:01.191429 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8847ce-f4bd-47f7-8fd2-ae8700e219a9 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:46:01.196139 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043c105-e7de-40d8-9257-5c83c02fb7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:46:01.196796 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043c105-e7de-40d8-9257-5c83c02fb7f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:46:01.197075 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043c105-e7de-40d8-9257-5c83c02fb7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:46:01.197347 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043c105-e7de-40d8-9257-5c83c02fb7f6 admin admin] PATH_INFO: `/v3/users/d81521fd864f4692bb3bab6ba7dfc8b9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:46:01.198456 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f043c105-e7de-40d8-9257-5c83c02fb7f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d81521fd864f4692bb3bab6ba7dfc8b9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:46:01.200374 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f043c105-e7de-40d8-9257-5c83c02fb7f6 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:46:01.245275 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f043c105-e7de-40d8-9257-5c83c02fb7f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd81521fd864f4692bb3bab6ba7dfc8b9'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:46:01.252698 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f043c105-e7de-40d8-9257-5c83c02fb7f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd81521fd864f4692bb3bab6ba7dfc8b9'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:46:01.264293 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f043c105-e7de-40d8-9257-5c83c02fb7f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd81521fd864f4692bb3bab6ba7dfc8b9'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:46:01.274460 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f043c105-e7de-40d8-9257-5c83c02fb7f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd81521fd864f4692bb3bab6ba7dfc8b9'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:46:01.275127 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f043c105-e7de-40d8-9257-5c83c02fb7f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd81521fd864f4692bb3bab6ba7dfc8b9'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:46:01.281978 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1285/2570] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:46:01 2022] DELETE /identity/v3/users/d81521fd864f4692bb3bab6ba7dfc8b9 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:46:01.287983 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443020e-541e-4e16-bd12-e744cda10da4 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:46:01.288465 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443020e-541e-4e16-bd12-e744cda10da4 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:46:01.288617 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443020e-541e-4e16-bd12-e744cda10da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:46:01.307731 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-1443020e-541e-4e16-bd12-e744cda10da4 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:46:01.308948 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1443020e-541e-4e16-bd12-e744cda10da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:46:01.368242 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1286/2571] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:46:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:46:01.378804 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043c105-e7de-40d8-9257-5c83c02fb7f6 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:46:01.380227 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f043c105-e7de-40d8-9257-5c83c02fb7f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:46:01.438183 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043c105-e7de-40d8-9257-5c83c02fb7f6 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:46:01.442038 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a1aa8c-fc27-41f7-ade7-3c229012ece4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:46:01.442648 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1aa8c-fc27-41f7-ade7-3c229012ece4 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:46:01.442924 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1aa8c-fc27-41f7-ade7-3c229012ece4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:46:01.443193 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1aa8c-fc27-41f7-ade7-3c229012ece4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:46:01.452872 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a1aa8c-fc27-41f7-ade7-3c229012ece4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:46:01.454326 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a1aa8c-fc27-41f7-ade7-3c229012ece4 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:46:01.464121 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1286/2572] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:46:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:46:01.728679 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1443020e-541e-4e16-bd12-e744cda10da4 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:46:01.730113 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1443020e-541e-4e16-bd12-e744cda10da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:46:01.742654 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1443020e-541e-4e16-bd12-e744cda10da4 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:46:01.747480 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f39975-0128-47f6-8dd5-0f2441de9971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:46:01.748752 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f39975-0128-47f6-8dd5-0f2441de9971 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:46:01.749035 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f39975-0128-47f6-8dd5-0f2441de9971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:46:01.749307 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f39975-0128-47f6-8dd5-0f2441de9971 admin admin] PATH_INFO: `/v3/projects/6cc3a300905949c69fed9679edb69e6c` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:46:01.750437 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f39975-0128-47f6-8dd5-0f2441de9971 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cc3a300905949c69fed9679edb69e6c)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:46:01.752340 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f39975-0128-47f6-8dd5-0f2441de9971 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:46:01.799751 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a3f39975-0128-47f6-8dd5-0f2441de9971 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cc3a300905949c69fed9679edb69e6c', 'request_id': 'req-a3f39975-0128-47f6-8dd5-0f2441de9971'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:46:01.805294 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a3f39975-0128-47f6-8dd5-0f2441de9971 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cc3a300905949c69fed9679edb69e6c', 'request_id': 'req-a3f39975-0128-47f6-8dd5-0f2441de9971'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:46:01.812552 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a3f39975-0128-47f6-8dd5-0f2441de9971 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cc3a300905949c69fed9679edb69e6c', 'request_id': 'req-a3f39975-0128-47f6-8dd5-0f2441de9971'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:46:01.823805 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a3f39975-0128-47f6-8dd5-0f2441de9971 admin admin] The token cache is being invalidate because project 6cc3a300905949c69fed9679edb69e6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:46:01.824120 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a3f39975-0128-47f6-8dd5-0f2441de9971 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:46:01.824828 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a3f39975-0128-47f6-8dd5-0f2441de9971 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:46:01.826258 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1287/2573] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:46:01 2022] DELETE /identity/v3/projects/6cc3a300905949c69fed9679edb69e6c => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:46:01.831532 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168e2b2-ecd5-462f-ac59-cc90e0f8cf9c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:46:01.831819 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168e2b2-ecd5-462f-ac59-cc90e0f8cf9c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:46:01.832122 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168e2b2-ecd5-462f-ac59-cc90e0f8cf9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:46:01.852105 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-4168e2b2-ecd5-462f-ac59-cc90e0f8cf9c None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:46:01.853401 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4168e2b2-ecd5-462f-ac59-cc90e0f8cf9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:46:01.887618 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:46:01.915339 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1287/2574] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:46:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:46:01.923090 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a5ae2-1f0f-4fe9-8dbd-9cbbadd0444d None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:46:01.923378 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a5ae2-1f0f-4fe9-8dbd-9cbbadd0444d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:46:01.923662 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a5ae2-1f0f-4fe9-8dbd-9cbbadd0444d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:46:01.943302 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-cc6a5ae2-1f0f-4fe9-8dbd-9cbbadd0444d None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:46:01.944497 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-cc6a5ae2-1f0f-4fe9-8dbd-9cbbadd0444d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:46:01.953181 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1288/2575] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:46:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:46:01.957156 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168e2b2-ecd5-462f-ac59-cc90e0f8cf9c None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:46:01.966649 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168e2b2-ecd5-462f-ac59-cc90e0f8cf9c None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:46:01.971907 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23473f7a-20cf-4e37-a953-072e6d7ec045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:46:01.972492 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23473f7a-20cf-4e37-a953-072e6d7ec045 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:46:01.972799 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23473f7a-20cf-4e37-a953-072e6d7ec045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:46:01.973071 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23473f7a-20cf-4e37-a953-072e6d7ec045 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:46:01.973709 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23473f7a-20cf-4e37-a953-072e6d7ec045 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 09:46:01.974060 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23473f7a-20cf-4e37-a953-072e6d7ec045 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:46:01.976173 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23473f7a-20cf-4e37-a953-072e6d7ec045 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:46:01.987861 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1288/2576] 10.222.0.70 () {58 vars in 1153 bytes} [Thu Oct 27 09:46:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 09:46:01.991942 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6a5ae2-1f0f-4fe9-8dbd-9cbbadd0444d None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:46:02.001848 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6a5ae2-1f0f-4fe9-8dbd-9cbbadd0444d None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:46:02.006492 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595ace32-9ad6-48c9-ae23-69cfd3d6d680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:46:02.007088 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ace32-9ad6-48c9-ae23-69cfd3d6d680 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:46:02.007369 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ace32-9ad6-48c9-ae23-69cfd3d6d680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:46:02.007639 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ace32-9ad6-48c9-ae23-69cfd3d6d680 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:46:02.008258 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595ace32-9ad6-48c9-ae23-69cfd3d6d680 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:46:02.010285 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595ace32-9ad6-48c9-ae23-69cfd3d6d680 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:46:02.037970 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1289/2577] 10.222.0.70 () {60 vars in 1153 bytes} [Thu Oct 27 09:46:01 2022] POST /identity/v3/projects => generated 399 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:46:02.042094 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23473f7a-20cf-4e37-a953-072e6d7ec045 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:46:02.051882 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23473f7a-20cf-4e37-a953-072e6d7ec045 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:46:02.108005 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24388cdc-43a1-4cf5-98fb-00eee4d3124d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:46:02.108743 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24388cdc-43a1-4cf5-98fb-00eee4d3124d admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:46:02.109022 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24388cdc-43a1-4cf5-98fb-00eee4d3124d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:46:02.109287 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24388cdc-43a1-4cf5-98fb-00eee4d3124d admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:46:02.109912 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24388cdc-43a1-4cf5-98fb-00eee4d3124d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:46:02.111794 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24388cdc-43a1-4cf5-98fb-00eee4d3124d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:46:02.131541 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1289/2578] 10.222.0.70 () {60 vars in 1144 bytes} [Thu Oct 27 09:46:02 2022] POST /identity/v3/users => generated 300 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 09:46:02.135570 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595ace32-9ad6-48c9-ae23-69cfd3d6d680 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:46:02.146345 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595ace32-9ad6-48c9-ae23-69cfd3d6d680 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:46:02.150814 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb11dc-1c02-4ce9-bca6-454245cc347e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:46:02.151398 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb11dc-1c02-4ce9-bca6-454245cc347e admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:46:02.151670 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb11dc-1c02-4ce9-bca6-454245cc347e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:46:02.151936 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb11dc-1c02-4ce9-bca6-454245cc347e admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:46:02.152719 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceb11dc-1c02-4ce9-bca6-454245cc347e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:46:02.154320 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceb11dc-1c02-4ce9-bca6-454245cc347e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:46:02.164967 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1290/2579] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:46:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:46:02.168780 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24388cdc-43a1-4cf5-98fb-00eee4d3124d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:46:02.178522 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24388cdc-43a1-4cf5-98fb-00eee4d3124d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:46:02.183170 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59795f9-82b7-42d6-bcbc-a4fa8456ac49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:46:02.183784 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59795f9-82b7-42d6-bcbc-a4fa8456ac49 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:46:02.184059 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59795f9-82b7-42d6-bcbc-a4fa8456ac49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:46:02.184328 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59795f9-82b7-42d6-bcbc-a4fa8456ac49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:46:02.184994 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59795f9-82b7-42d6-bcbc-a4fa8456ac49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:46:02.186790 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59795f9-82b7-42d6-bcbc-a4fa8456ac49 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:46:02.197468 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1290/2580] 10.222.0.70 () {58 vars in 1122 bytes} [Thu Oct 27 09:46:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 09:46:02.201544 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb11dc-1c02-4ce9-bca6-454245cc347e admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:46:02.211591 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb11dc-1c02-4ce9-bca6-454245cc347e admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:46:02.216241 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbb144a-7fb6-4a81-9527-af2dfb50ff49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:46:02.216917 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb144a-7fb6-4a81-9527-af2dfb50ff49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:46:02.217194 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb144a-7fb6-4a81-9527-af2dfb50ff49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:46:02.217465 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb144a-7fb6-4a81-9527-af2dfb50ff49 admin admin] PATH_INFO: `/v3/projects/bca965ae7f094593ba8a819e2be7f4b3/users/66b574310e5e44d0aa4ff3b2430d7215/roles/eea7cd23e01b4506a9eb6ab86f9adcd2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:46:02.220282 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:46:02.234686 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbb144a-7fb6-4a81-9527-af2dfb50ff49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bca965ae7f094593ba8a819e2be7f4b3, user_id=66b574310e5e44d0aa4ff3b2430d7215, role_id=eea7cd23e01b4506a9eb6ab86f9adcd2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:46:02.237346 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbb144a-7fb6-4a81-9527-af2dfb50ff49 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:46:02.249937 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1291/2581] 10.222.0.70 () {60 vars in 1483 bytes} [Thu Oct 27 09:46:02 2022] PUT /identity/v3/projects/bca965ae7f094593ba8a819e2be7f4b3/users/66b574310e5e44d0aa4ff3b2430d7215/roles/eea7cd23e01b4506a9eb6ab86f9adcd2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:46:02.261226 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0dfe9-dbb4-4a15-ada0-81e9990a90b1 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:46:02.261532 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0dfe9-dbb4-4a15-ada0-81e9990a90b1 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:46:02.261801 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0dfe9-dbb4-4a15-ada0-81e9990a90b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:46:02.290359 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-33e0dfe9-dbb4-4a15-ada0-81e9990a90b1 None None] MFA Rules not processed for user `66b574310e5e44d0aa4ff3b2430d7215`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:46:02.291563 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-33e0dfe9-dbb4-4a15-ada0-81e9990a90b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:46:02.354054 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1291/2582] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:46:02 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:46:02.365310 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbb144a-7fb6-4a81-9527-af2dfb50ff49 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:46:02.367022 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2cbb144a-7fb6-4a81-9527-af2dfb50ff49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:46:02.423336 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbb144a-7fb6-4a81-9527-af2dfb50ff49 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:46:02.426836 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790e3831-fa44-4a7d-b5be-67f2e3de243f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8e13ce967449c1b12783183e953579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:46:02.427423 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e3831-fa44-4a7d-b5be-67f2e3de243f service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:46:02.427696 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e3831-fa44-4a7d-b5be-67f2e3de243f service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:46:02.427961 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e3831-fa44-4a7d-b5be-67f2e3de243f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:46:02.436091 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790e3831-fa44-4a7d-b5be-67f2e3de243f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:46:02.438065 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790e3831-fa44-4a7d-b5be-67f2e3de243f service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:46:02.447098 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1292/2583] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:46:02 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:46:09.472557 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e0dfe9-dbb4-4a15-ada0-81e9990a90b1 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:46:09.511332 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:46:09.534871 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e0dfe9-dbb4-4a15-ada0-81e9990a90b1 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:46:09.539348 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b59e54-20a1-4b1b-8097-c11025251daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:46:09.539902 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b59e54-20a1-4b1b-8097-c11025251daa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:46:09.540179 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b59e54-20a1-4b1b-8097-c11025251daa admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:46:09.540444 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b59e54-20a1-4b1b-8097-c11025251daa admin admin] PATH_INFO: `/v3/users/66b574310e5e44d0aa4ff3b2430d7215` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:46:09.541571 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b59e54-20a1-4b1b-8097-c11025251daa admin admin] RBAC: Authorizing `identity:delete_user(user_id=66b574310e5e44d0aa4ff3b2430d7215)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:46:09.543466 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b59e54-20a1-4b1b-8097-c11025251daa admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:46:09.588684 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b9b59e54-20a1-4b1b-8097-c11025251daa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66b574310e5e44d0aa4ff3b2430d7215'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:46:09.595490 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b9b59e54-20a1-4b1b-8097-c11025251daa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66b574310e5e44d0aa4ff3b2430d7215'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:46:09.608229 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b9b59e54-20a1-4b1b-8097-c11025251daa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66b574310e5e44d0aa4ff3b2430d7215'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:46:09.615788 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b9b59e54-20a1-4b1b-8097-c11025251daa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66b574310e5e44d0aa4ff3b2430d7215'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:46:09.616384 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b9b59e54-20a1-4b1b-8097-c11025251daa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66b574310e5e44d0aa4ff3b2430d7215'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:46:09.622268 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1292/2584] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:46:09 2022] DELETE /identity/v3/users/66b574310e5e44d0aa4ff3b2430d7215 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:46:09.627381 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13bf8b-a464-4b93-84c6-40b4c43f7278 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:46:09.627862 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13bf8b-a464-4b93-84c6-40b4c43f7278 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:46:09.627971 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13bf8b-a464-4b93-84c6-40b4c43f7278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:46:09.646373 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-8e13bf8b-a464-4b93-84c6-40b4c43f7278 None None] MFA Rules not processed for user `4e3d5d12b2ba49459e93d0a715783a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:46:09.647571 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8e13bf8b-a464-4b93-84c6-40b4c43f7278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:46:09.704915 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1293/2585] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:46:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:46:09.714511 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b59e54-20a1-4b1b-8097-c11025251daa admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:46:09.715926 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b9b59e54-20a1-4b1b-8097-c11025251daa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:46:09.772631 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b59e54-20a1-4b1b-8097-c11025251daa admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:46:09.776250 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f211853c-7c91-4e1d-86d8-d18b0e2b1bf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:46:09.776878 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f211853c-7c91-4e1d-86d8-d18b0e2b1bf7 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:46:09.777157 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f211853c-7c91-4e1d-86d8-d18b0e2b1bf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:46:09.777427 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f211853c-7c91-4e1d-86d8-d18b0e2b1bf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:46:09.786949 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f211853c-7c91-4e1d-86d8-d18b0e2b1bf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:46:09.788376 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f211853c-7c91-4e1d-86d8-d18b0e2b1bf7 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:46:09.798848 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1293/2586] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:46:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:46:10.061941 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e13bf8b-a464-4b93-84c6-40b4c43f7278 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:46:10.063357 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8e13bf8b-a464-4b93-84c6-40b4c43f7278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:46:10.075671 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e13bf8b-a464-4b93-84c6-40b4c43f7278 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:46:10.081212 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9e64c-15ec-477b-8ab6-f68219d67f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:46:10.081827 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9e64c-15ec-477b-8ab6-f68219d67f4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:46:10.082107 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9e64c-15ec-477b-8ab6-f68219d67f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:46:10.082381 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9e64c-15ec-477b-8ab6-f68219d67f4c admin admin] PATH_INFO: `/v3/projects/bca965ae7f094593ba8a819e2be7f4b3` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:46:10.083512 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def9e64c-15ec-477b-8ab6-f68219d67f4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bca965ae7f094593ba8a819e2be7f4b3)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:46:10.085454 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def9e64c-15ec-477b-8ab6-f68219d67f4c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:46:10.132343 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-def9e64c-15ec-477b-8ab6-f68219d67f4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bca965ae7f094593ba8a819e2be7f4b3', 'request_id': 'req-def9e64c-15ec-477b-8ab6-f68219d67f4c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:46:10.137972 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-def9e64c-15ec-477b-8ab6-f68219d67f4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bca965ae7f094593ba8a819e2be7f4b3', 'request_id': 'req-def9e64c-15ec-477b-8ab6-f68219d67f4c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:46:10.145558 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-def9e64c-15ec-477b-8ab6-f68219d67f4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bca965ae7f094593ba8a819e2be7f4b3', 'request_id': 'req-def9e64c-15ec-477b-8ab6-f68219d67f4c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:46:10.157244 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-def9e64c-15ec-477b-8ab6-f68219d67f4c admin admin] The token cache is being invalidate because project bca965ae7f094593ba8a819e2be7f4b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:46:10.157530 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-def9e64c-15ec-477b-8ab6-f68219d67f4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:46:10.158227 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-def9e64c-15ec-477b-8ab6-f68219d67f4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:46:10.159729 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1294/2587] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:46:10 2022] DELETE /identity/v3/projects/bca965ae7f094593ba8a819e2be7f4b3 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:46:19.793830 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=63,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:46:20.153611 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:46:36.655030 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f211853c-7c91-4e1d-86d8-d18b0e2b1bf7 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:46:36.656985 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f211853c-7c91-4e1d-86d8-d18b0e2b1bf7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:46:36.667904 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f211853c-7c91-4e1d-86d8-d18b0e2b1bf7 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:46:36.716210 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c31317c-0b85-491f-9d63-2f2b938b3378 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:46:36.716852 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31317c-0b85-491f-9d63-2f2b938b3378 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:46:36.717131 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31317c-0b85-491f-9d63-2f2b938b3378 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:46:36.717403 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31317c-0b85-491f-9d63-2f2b938b3378 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:46:36.719162 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5c31317c-0b85-491f-9d63-2f2b938b3378 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:46:36.799205 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c31317c-0b85-491f-9d63-2f2b938b3378 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:46:36.800677 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c31317c-0b85-491f-9d63-2f2b938b3378 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:46:36.821078 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1294/2588] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:46:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:46:38.010917 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9e64c-15ec-477b-8ab6-f68219d67f4c admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:46:38.012307 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-def9e64c-15ec-477b-8ab6-f68219d67f4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:46:38.024360 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9e64c-15ec-477b-8ab6-f68219d67f4c admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:46:38.027855 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2963d45a-59b7-4f0f-b527-ab68212a5f1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:46:38.028465 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2963d45a-59b7-4f0f-b527-ab68212a5f1e service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:46:38.028773 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2963d45a-59b7-4f0f-b527-ab68212a5f1e service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:46:38.029037 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2963d45a-59b7-4f0f-b527-ab68212a5f1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:46:38.030690 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2963d45a-59b7-4f0f-b527-ab68212a5f1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:46:38.043552 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2963d45a-59b7-4f0f-b527-ab68212a5f1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:46:38.045274 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2963d45a-59b7-4f0f-b527-ab68212a5f1e service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:46:38.056443 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1295/2589] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:46:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:46:40.369361 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81b09b-d239-41ea-a8c2-6691887699ab None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:46:40.369830 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81b09b-d239-41ea-a8c2-6691887699ab None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:46:40.369933 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81b09b-d239-41ea-a8c2-6691887699ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:46:40.389211 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-fb81b09b-d239-41ea-a8c2-6691887699ab None None] MFA Rules not processed for user `127c4fdd8f2b40ae8d98208d64a7f20d`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:46:40.390402 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fb81b09b-d239-41ea-a8c2-6691887699ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:46:40.444889 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1295/2590] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:46:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:46:48.051691 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=6 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:46:50.438336 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=88 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:47:44.795823 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2963d45a-59b7-4f0f-b527-ab68212a5f1e service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:47:44.805338 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2963d45a-59b7-4f0f-b527-ab68212a5f1e service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:47:44.809123 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc22fea-86a5-4058-a877-53787d11d63b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:47:44.809756 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc22fea-86a5-4058-a877-53787d11d63b service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:47:44.810052 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc22fea-86a5-4058-a877-53787d11d63b service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:47:44.810316 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc22fea-86a5-4058-a877-53787d11d63b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:47:44.812704 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4dc22fea-86a5-4058-a877-53787d11d63b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:47:44.822384 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc22fea-86a5-4058-a877-53787d11d63b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:47:44.823787 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc22fea-86a5-4058-a877-53787d11d63b service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:47:44.831724 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1296/2591] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:47:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:47:54.828155 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=6 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:48:32.270917 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08222755-a777-43e7-a201-314d08567732 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:48:32.271456 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08222755-a777-43e7-a201-314d08567732 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:48:32.271456 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08222755-a777-43e7-a201-314d08567732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:48:32.290919 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-08222755-a777-43e7-a201-314d08567732 None None] MFA Rules not processed for user `127c4fdd8f2b40ae8d98208d64a7f20d`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 09:48:32.292101 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-08222755-a777-43e7-a201-314d08567732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:48:32.298812 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1296/2592] 10.222.0.70 () {58 vars in 958 bytes} [Thu Oct 27 09:48:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 09:48:32.616025 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc22fea-86a5-4058-a877-53787d11d63b service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:48:32.624450 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc22fea-86a5-4058-a877-53787d11d63b service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:48:32.628640 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5667940a-2a7f-4347-9c94-a29d1f2d24ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:48:32.629225 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5667940a-2a7f-4347-9c94-a29d1f2d24ab service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:48:32.629496 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5667940a-2a7f-4347-9c94-a29d1f2d24ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:48:32.629759 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5667940a-2a7f-4347-9c94-a29d1f2d24ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:48:32.631452 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5667940a-2a7f-4347-9c94-a29d1f2d24ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:48:32.640649 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5667940a-2a7f-4347-9c94-a29d1f2d24ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:48:32.642088 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5667940a-2a7f-4347-9c94-a29d1f2d24ab service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:48:32.650481 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1297/2593] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:48:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:48:33.094596 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08222755-a777-43e7-a201-314d08567732 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:48:33.096282 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-08222755-a777-43e7-a201-314d08567732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:48:33.166326 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08222755-a777-43e7-a201-314d08567732 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:48:33.171238 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40df6ec-c0ad-4e92-b2e5-742c6a30ffcc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2316b30a6e7b4175bd7b90583584eab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:48:33.171836 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40df6ec-c0ad-4e92-b2e5-742c6a30ffcc service placement] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:48:33.172111 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40df6ec-c0ad-4e92-b2e5-742c6a30ffcc service placement] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:48:33.172377 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40df6ec-c0ad-4e92-b2e5-742c6a30ffcc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:48:33.174315 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d40df6ec-c0ad-4e92-b2e5-742c6a30ffcc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:48:33.185530 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40df6ec-c0ad-4e92-b2e5-742c6a30ffcc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:48:33.187118 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40df6ec-c0ad-4e92-b2e5-742c6a30ffcc service placement] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:48:33.198173 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1297/2594] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:48:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:48:35.601430 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5667940a-2a7f-4347-9c94-a29d1f2d24ab service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:48:35.603910 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5667940a-2a7f-4347-9c94-a29d1f2d24ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:48:35.623333 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5667940a-2a7f-4347-9c94-a29d1f2d24ab service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:48:35.630013 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95393b1f-0e26-4469-8296-9a232f577030 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe4a6e4b6184826b30ac4fc13ad9b1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:48:35.630791 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95393b1f-0e26-4469-8296-9a232f577030 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:48:35.631167 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95393b1f-0e26-4469-8296-9a232f577030 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:48:35.631568 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95393b1f-0e26-4469-8296-9a232f577030 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:48:35.633438 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-95393b1f-0e26-4469-8296-9a232f577030 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:48:35.646948 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95393b1f-0e26-4469-8296-9a232f577030 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:48:35.648699 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95393b1f-0e26-4469-8296-9a232f577030 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:48:35.660005 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1298/2595] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:48:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:48:43.193635 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=35 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:48:45.655088 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=12 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:48:46.086848 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40df6ec-c0ad-4e92-b2e5-742c6a30ffcc service placement] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:48:46.088621 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d40df6ec-c0ad-4e92-b2e5-742c6a30ffcc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:48:46.105158 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40df6ec-c0ad-4e92-b2e5-742c6a30ffcc service placement] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:48:46.110858 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d97b339-b932-480a-9140-5c6dca951269 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe4a6e4b6184826b30ac4fc13ad9b1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:48:46.111487 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d97b339-b932-480a-9140-5c6dca951269 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:48:46.111788 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d97b339-b932-480a-9140-5c6dca951269 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:48:46.112108 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d97b339-b932-480a-9140-5c6dca951269 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:48:46.114426 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6d97b339-b932-480a-9140-5c6dca951269 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:48:46.126692 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d97b339-b932-480a-9140-5c6dca951269 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:48:46.128301 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d97b339-b932-480a-9140-5c6dca951269 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:48:46.139290 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1298/2596] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:48:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:48:50.829764 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95393b1f-0e26-4469-8296-9a232f577030 service nova] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:48:50.841607 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95393b1f-0e26-4469-8296-9a232f577030 service nova] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:48:50.848149 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588c697b-61e0-4e9f-87da-dfc4c971b073 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe4a6e4b6184826b30ac4fc13ad9b1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:48:50.848719 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c697b-61e0-4e9f-87da-dfc4c971b073 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:48:50.848997 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c697b-61e0-4e9f-87da-dfc4c971b073 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:48:50.849266 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c697b-61e0-4e9f-87da-dfc4c971b073 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:48:50.851092 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-588c697b-61e0-4e9f-87da-dfc4c971b073 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:48:50.863560 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588c697b-61e0-4e9f-87da-dfc4c971b073 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:48:50.865161 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588c697b-61e0-4e9f-87da-dfc4c971b073 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:48:50.876040 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1299/2597] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:48:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:48:56.134388 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=6 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:49:00.871119 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=6 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:49:50.071155 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d97b339-b932-480a-9140-5c6dca951269 service nova] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:49:50.072581 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6d97b339-b932-480a-9140-5c6dca951269 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:49:50.087384 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d97b339-b932-480a-9140-5c6dca951269 service nova] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:49:50.092307 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30cab0-3110-4cfb-9d97-f4a546202165 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2316b30a6e7b4175bd7b90583584eab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:49:50.092938 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30cab0-3110-4cfb-9d97-f4a546202165 service placement] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:49:50.093215 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30cab0-3110-4cfb-9d97-f4a546202165 service placement] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:49:50.093484 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30cab0-3110-4cfb-9d97-f4a546202165 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:49:50.095164 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-aa30cab0-3110-4cfb-9d97-f4a546202165 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:49:50.107264 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa30cab0-3110-4cfb-9d97-f4a546202165 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:49:50.108914 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa30cab0-3110-4cfb-9d97-f4a546202165 service placement] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:49:50.119044 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1299/2598] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:49:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:50:00.114524 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=6 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:50:13.070070 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588c697b-61e0-4e9f-87da-dfc4c971b073 service nova] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:50:13.079653 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588c697b-61e0-4e9f-87da-dfc4c971b073 service nova] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:50:13.083317 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1943eaf5-f65e-4ff0-9e3a-1f825ef1862d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:50:13.083892 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1943eaf5-f65e-4ff0-9e3a-1f825ef1862d service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:50:13.084170 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1943eaf5-f65e-4ff0-9e3a-1f825ef1862d service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:50:13.084439 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1943eaf5-f65e-4ff0-9e3a-1f825ef1862d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:50:13.086149 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1943eaf5-f65e-4ff0-9e3a-1f825ef1862d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:50:13.096773 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1943eaf5-f65e-4ff0-9e3a-1f825ef1862d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:50:13.098212 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1943eaf5-f65e-4ff0-9e3a-1f825ef1862d service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:50:13.107497 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1300/2599] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:50:13 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:50:23.103400 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=6 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:50:26.389138 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30cab0-3110-4cfb-9d97-f4a546202165 service placement] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:50:26.390474 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-aa30cab0-3110-4cfb-9d97-f4a546202165 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:50:26.402342 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30cab0-3110-4cfb-9d97-f4a546202165 service placement] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:50:26.405871 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1a673-7d25-4fb1-8030-7dfa070797bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45d8920d9a7940abb112dbbb3d7f6e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:50:26.406453 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1a673-7d25-4fb1-8030-7dfa070797bc service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:50:26.406723 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1a673-7d25-4fb1-8030-7dfa070797bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:50:26.406985 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1a673-7d25-4fb1-8030-7dfa070797bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:50:26.408627 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8cb1a673-7d25-4fb1-8030-7dfa070797bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:50:26.472609 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb1a673-7d25-4fb1-8030-7dfa070797bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:50:26.474048 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb1a673-7d25-4fb1-8030-7dfa070797bc service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:50:26.483620 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1300/2600] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:50:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:50:29.274865 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1943eaf5-f65e-4ff0-9e3a-1f825ef1862d service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:50:29.289242 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1943eaf5-f65e-4ff0-9e3a-1f825ef1862d service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:50:29.294940 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0ca461-942a-4b35-8ef7-0a37d3f71dee service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2316b30a6e7b4175bd7b90583584eab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:50:29.295559 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ca461-942a-4b35-8ef7-0a37d3f71dee service placement] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:50:29.295832 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ca461-942a-4b35-8ef7-0a37d3f71dee service placement] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:50:29.296287 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ca461-942a-4b35-8ef7-0a37d3f71dee service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:50:29.298048 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-bf0ca461-942a-4b35-8ef7-0a37d3f71dee service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:50:29.311380 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0ca461-942a-4b35-8ef7-0a37d3f71dee service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:50:29.313344 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0ca461-942a-4b35-8ef7-0a37d3f71dee service placement] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:50:29.330745 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1301/2601] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:50:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:50:32.662839 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1a673-7d25-4fb1-8030-7dfa070797bc service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:50:32.674051 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1a673-7d25-4fb1-8030-7dfa070797bc service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:50:32.679202 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2951b5e-e12c-4330-98a8-1fff5dacb26c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2316b30a6e7b4175bd7b90583584eab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:50:32.679881 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2951b5e-e12c-4330-98a8-1fff5dacb26c service placement] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:50:32.680226 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2951b5e-e12c-4330-98a8-1fff5dacb26c service placement] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:50:32.680638 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2951b5e-e12c-4330-98a8-1fff5dacb26c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:50:32.682478 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e2951b5e-e12c-4330-98a8-1fff5dacb26c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:50:32.694384 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2951b5e-e12c-4330-98a8-1fff5dacb26c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:50:32.696650 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2951b5e-e12c-4330-98a8-1fff5dacb26c service placement] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:50:32.709063 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1301/2602] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:50:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:50:32.723931 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0ca461-942a-4b35-8ef7-0a37d3f71dee service placement] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:50:32.725570 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-bf0ca461-942a-4b35-8ef7-0a37d3f71dee service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:50:32.748031 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0ca461-942a-4b35-8ef7-0a37d3f71dee service placement] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:50:32.752758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef83d789-3551-4009-8249-977968584c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:50:32.753388 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83d789-3551-4009-8249-977968584c14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:50:32.753667 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83d789-3551-4009-8249-977968584c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:50:32.753936 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83d789-3551-4009-8249-977968584c14 admin admin] PATH_INFO: `/v3/users/127c4fdd8f2b40ae8d98208d64a7f20d` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:50:32.755091 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef83d789-3551-4009-8249-977968584c14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=127c4fdd8f2b40ae8d98208d64a7f20d)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:50:32.757076 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef83d789-3551-4009-8249-977968584c14 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:50:32.803264 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ef83d789-3551-4009-8249-977968584c14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '127c4fdd8f2b40ae8d98208d64a7f20d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:50:32.810837 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ef83d789-3551-4009-8249-977968584c14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '127c4fdd8f2b40ae8d98208d64a7f20d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:50:32.823365 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ef83d789-3551-4009-8249-977968584c14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '127c4fdd8f2b40ae8d98208d64a7f20d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:50:32.831151 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ef83d789-3551-4009-8249-977968584c14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '127c4fdd8f2b40ae8d98208d64a7f20d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:50:32.831796 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ef83d789-3551-4009-8249-977968584c14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '127c4fdd8f2b40ae8d98208d64a7f20d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:50:32.837758 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1302/2603] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:50:32 2022] DELETE /identity/v3/users/127c4fdd8f2b40ae8d98208d64a7f20d => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:50:33.008588 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2951b5e-e12c-4330-98a8-1fff5dacb26c service placement] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:50:33.019796 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2951b5e-e12c-4330-98a8-1fff5dacb26c service placement] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:50:33.076611 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b40c3b-f546-455b-aea2-f87411d9e656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:50:33.077238 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b40c3b-f546-455b-aea2-f87411d9e656 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:50:33.077512 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b40c3b-f546-455b-aea2-f87411d9e656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:50:33.077782 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b40c3b-f546-455b-aea2-f87411d9e656 admin admin] PATH_INFO: `/v3/projects/4ee2066bcc7240f9a63d5c11a2b04a18` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:50:33.078885 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b40c3b-f546-455b-aea2-f87411d9e656 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ee2066bcc7240f9a63d5c11a2b04a18)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:50:33.080822 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b40c3b-f546-455b-aea2-f87411d9e656 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:50:33.126193 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c3b40c3b-f546-455b-aea2-f87411d9e656 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ee2066bcc7240f9a63d5c11a2b04a18', 'request_id': 'req-c3b40c3b-f546-455b-aea2-f87411d9e656'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:50:33.132336 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c3b40c3b-f546-455b-aea2-f87411d9e656 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ee2066bcc7240f9a63d5c11a2b04a18', 'request_id': 'req-c3b40c3b-f546-455b-aea2-f87411d9e656'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:50:33.138276 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c3b40c3b-f546-455b-aea2-f87411d9e656 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ee2066bcc7240f9a63d5c11a2b04a18', 'request_id': 'req-c3b40c3b-f546-455b-aea2-f87411d9e656'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:50:33.149850 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c3b40c3b-f546-455b-aea2-f87411d9e656 admin admin] The token cache is being invalidate because project 4ee2066bcc7240f9a63d5c11a2b04a18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:50:33.150136 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c3b40c3b-f546-455b-aea2-f87411d9e656 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:50:33.150759 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c3b40c3b-f546-455b-aea2-f87411d9e656 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:50:33.152230 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1302/2604] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:50:33 2022] DELETE /identity/v3/projects/4ee2066bcc7240f9a63d5c11a2b04a18 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:50:33.156345 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef83d789-3551-4009-8249-977968584c14 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:50:33.157814 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ef83d789-3551-4009-8249-977968584c14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:50:33.171108 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef83d789-3551-4009-8249-977968584c14 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:50:33.229908 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1027bb8-fba7-48e8-aac9-8e4126d2ac2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:50:33.230465 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1027bb8-fba7-48e8-aac9-8e4126d2ac2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:50:33.230748 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1027bb8-fba7-48e8-aac9-8e4126d2ac2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:50:33.231021 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1027bb8-fba7-48e8-aac9-8e4126d2ac2c admin admin] PATH_INFO: `/v3/users/df8e713098ea4f0dad0d9914cca6e44b` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:50:33.232186 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1027bb8-fba7-48e8-aac9-8e4126d2ac2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=df8e713098ea4f0dad0d9914cca6e44b)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:50:33.234155 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1027bb8-fba7-48e8-aac9-8e4126d2ac2c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:50:33.280838 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-c1027bb8-fba7-48e8-aac9-8e4126d2ac2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df8e713098ea4f0dad0d9914cca6e44b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:50:33.287883 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-c1027bb8-fba7-48e8-aac9-8e4126d2ac2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df8e713098ea4f0dad0d9914cca6e44b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:50:33.292827 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:50:33.302964 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-c1027bb8-fba7-48e8-aac9-8e4126d2ac2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df8e713098ea4f0dad0d9914cca6e44b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:50:33.311239 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-c1027bb8-fba7-48e8-aac9-8e4126d2ac2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df8e713098ea4f0dad0d9914cca6e44b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:50:33.311884 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-c1027bb8-fba7-48e8-aac9-8e4126d2ac2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df8e713098ea4f0dad0d9914cca6e44b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:50:33.317614 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1303/2605] 10.222.0.70 () {58 vars in 1224 bytes} [Thu Oct 27 09:50:33 2022] DELETE /identity/v3/users/df8e713098ea4f0dad0d9914cca6e44b => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:50:33.486370 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b40c3b-f546-455b-aea2-f87411d9e656 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:50:33.487824 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c3b40c3b-f546-455b-aea2-f87411d9e656 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:50:33.501846 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b40c3b-f546-455b-aea2-f87411d9e656 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:50:33.509851 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:50:33.561420 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0edc187-2c9a-4dc7-b19c-431713959069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3d5d12b2ba49459e93d0a715783a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f8e01e4d9d43978ddeb5033ead57f5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:50:33.562043 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0edc187-2c9a-4dc7-b19c-431713959069 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:50:33.562319 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0edc187-2c9a-4dc7-b19c-431713959069 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:50:33.562588 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0edc187-2c9a-4dc7-b19c-431713959069 admin admin] PATH_INFO: `/v3/projects/5ffb0db501244b93bc6260ea11738c79` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:50:33.563682 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0edc187-2c9a-4dc7-b19c-431713959069 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ffb0db501244b93bc6260ea11738c79)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:50:33.565630 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0edc187-2c9a-4dc7-b19c-431713959069 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:50:33.611064 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d0edc187-2c9a-4dc7-b19c-431713959069 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ffb0db501244b93bc6260ea11738c79', 'request_id': 'req-d0edc187-2c9a-4dc7-b19c-431713959069'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:50:33.617070 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d0edc187-2c9a-4dc7-b19c-431713959069 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ffb0db501244b93bc6260ea11738c79', 'request_id': 'req-d0edc187-2c9a-4dc7-b19c-431713959069'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:50:33.623045 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d0edc187-2c9a-4dc7-b19c-431713959069 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ffb0db501244b93bc6260ea11738c79', 'request_id': 'req-d0edc187-2c9a-4dc7-b19c-431713959069'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:50:33.634616 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d0edc187-2c9a-4dc7-b19c-431713959069 admin admin] The token cache is being invalidate because project 5ffb0db501244b93bc6260ea11738c79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 09:50:33.634980 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d0edc187-2c9a-4dc7-b19c-431713959069 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:50:33.636109 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d0edc187-2c9a-4dc7-b19c-431713959069 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 09:50:33.637609 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1303/2606] 10.222.0.70 () {58 vars in 1233 bytes} [Thu Oct 27 09:50:33 2022] DELETE /identity/v3/projects/5ffb0db501244b93bc6260ea11738c79 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 09:50:43.314175 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:DELETE=3,keystone:SELECT=6,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:50:43.631618 dv-cinder-797970-7-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 09:54:56.077299 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1027bb8-fba7-48e8-aac9-8e4126d2ac2c admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 09:54:56.078695 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c1027bb8-fba7-48e8-aac9-8e4126d2ac2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:54:56.217008 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1027bb8-fba7-48e8-aac9-8e4126d2ac2c admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 09:54:56.279825 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a27cd46-2c1b-4a29-bc02-1fd1365a3a67 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2316b30a6e7b4175bd7b90583584eab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16ac205ec6f241d69d5e6b8408f09f38', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 09:54:56.280499 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a27cd46-2c1b-4a29-bc02-1fd1365a3a67 service placement] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 09:54:56.280806 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a27cd46-2c1b-4a29-bc02-1fd1365a3a67 service placement] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 09:54:56.281076 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a27cd46-2c1b-4a29-bc02-1fd1365a3a67 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 09:54:56.282776 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7a27cd46-2c1b-4a29-bc02-1fd1365a3a67 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 09:54:56.294396 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a27cd46-2c1b-4a29-bc02-1fd1365a3a67 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 09:54:56.295981 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a27cd46-2c1b-4a29-bc02-1fd1365a3a67 service placement] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 09:54:56.306389 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1304/2607] 10.222.0.70 () {60 vars in 1365 bytes} [Thu Oct 27 09:54:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 09:55:06.301565 dv-cinder-797970-7-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}